Apple study reveals major AI flaw in OpenAI, Google, and Meta LLMs

Massive Language Fashions (LLMs) is probably not as good as they appear, in response to a examine from Apple researchers.

LLMs from OpenAI, Google, Meta, and others have been touted for his or her spectacular reasoning abilities. However analysis suggests their purported intelligence could also be nearer to “subtle sample matching” than “true logical reasoning.” Yep, even OpenAI’s o1 superior reasoning mannequin.

The most typical benchmark for reasoning abilities is a take a look at referred to as GSM8K, however since it is so widespread, there is a threat of information contamination. Meaning LLMs may know the solutions to the take a look at as a result of they have been educated on these solutions, not due to their inherent intelligence.

To check this, the examine developed a brand new benchmark referred to as GSM-Symbolic which retains the essence of the reasoning issues, however adjustments the variables, like names, numbers, complexity, and including irrelevant data. What they found was stunning “fragility” in LLM efficiency. The examine examined over 20 fashions together with OpenAI’s o1 and GPT-4o, Google’s Gemma 2, and Meta’s Llama 3. With each single mannequin, the mannequin’s efficiency decreased when the variables have been modified.

Accuracy decreased by just a few share factors when names and variables have been modified. And because the researchers famous, OpenAI’s fashions carried out higher than the opposite open-source fashions. Nevertheless the variance was deemed “non-negligible,” that means any actual variance should not have occurred. Nevertheless, issues received actually fascinating when researchers added “seemingly related however in the end inconsequential statements” to the combination.

To check the speculation that LLMs relied extra on sample matching than precise reasoning, the examine added superfluous phrases to math issues to see how the fashions would react. For instance, “Oliver picks 44 kiwis on Friday. Then he picks 58 kiwis on Saturday. On Sunday, he picks double the variety of kiwis he did on Friday, however 5 of them have been a bit smaller than common. What number of kiwis does Oliver have?”

What resulted was a major drop in efficiency throughout the board. OpenAI’s o1 Preview fared the most effective, with a drop of 17.5 p.c accuracy. That is nonetheless fairly dangerous, however not as dangerous as Microsoft’s Phi 3 mannequin which carried out 65 p.c worse.

Within the kiwi instance, the examine mentioned LLMs tended to subtract the 5 smaller kiwis from the equation with out understanding that kiwi measurement was irrelevant to the issue. This means that “fashions are inclined to convert statements to operations with out actually understanding their that means” which validates the researchers’ speculation that LLMs search for patterns in reasoning issues, somewhat than innately perceive the idea.

The examine did not mince phrases about its findings. Testing fashions’ on the benchmark that features irrelevant data “exposes a crucial flaw in LLMs’ capability to genuinely perceive mathematical ideas and discern related data for problem-solving.” Nevertheless, it bears mentioning that the authors of this examine work for Apple which is clearly a significant competitor with Google, Meta, and even OpenAI — though Apple and OpenAI have a partnership, Apple can be working by itself AI fashions.

That mentioned, the LLMs’ obvious lack of formal reasoning abilities cannot be ignored. In the end, it is a good reminder to mood AI hype with wholesome skepticism.