iPhone 16 Pro Max Shows Numerical Errors in MLX LLM Execution Compared to iPhone 15 Pro
By
rafaelcosta
3mo ago· 9 min readenInsight
90/100
Golden Brown
Bagelometer↗
Front-window bakery material. Catches the eye, delivers the goods.
Score90TypeanalysisSentimentnegative
Summary
The article details a technical investigation where the author discovered that their iPhone 16 Pro Max produces incorrect numerical outputs when running MLX LLMs (machine learning language models), while an iPhone 15 Pro and MacBook Pro run the same code correctly. The author suspects a hardware defect in the Neural Engine or other ML-related systems, documenting their debugging process and findings in a blog post format.
Key quotes
· 4 pulledMy iPhone 16 Pro Max produces garbage output when running MLX LLMs. An iPhone 15 Pro runs the same code perfectly. A MacBook Pro also runs the same code perfectly.
The tensor outputs on the 16 show numerical values an order of magnitude wrong.
I suspect it points to a hardware defect in the Neural Engine or some other ML-needed system.
It was a PITA to debug, but at least I got a blog post out of it.
TL;DR:
My iPhone 16 Pro Max produces garbage output when running MLX LLMs. An iPhone 15 Pro runs the same code perfectly. A MacBook Pro also runs the same code perfectly. The tensor outputs on the 16 show numerical values an order of magnitude wrong. I s
