Show a multimodal LLM a chemical structure diagram. Ask it to identify the molecule. It fails. Now ask it to reason about the molecule's properties given its name. It succeeds.
Li et al. evaluate leading multimodal models across five symbolic domains — language, culture, mathematics, physics, chemistry — and find a systematic cognitive mismatch: models fail at basic symbol recognition but succeed at complex reasoning tasks that depend on those same symbols.
The explanation: the models are not seeing the symbols at all. They are pattern-matching the visual input to a textual approximation, then reasoning over the text. When the visual-to-text mapping works (the symbol looks like something in training data), recognition succeeds. When it doesn't (unusual notation, handwritten formulas, culturally specific characters), recognition fails. But the reasoning that follows — given the correct name or description — remains strong because it operates entirely in language space.
The mismatch reveals a dissociation between perception and cognition in multimodal AI. The vision encoder extracts features. The language model reasons over features. But the features extracted from discrete symbols — mathematical formulas, chemical diagrams, musical notation — are unreliable. The models' linguistic probability patterns fill in what visual perception fails to provide, producing correct reasoning from incorrect perception.
The structural implication: multimodal LLMs have not learned to see symbols. They have learned to guess what symbols say based on visual context, then reason as if the guess were correct. When the guess is right, the system appears competent. When it is wrong, the error is invisible — the reasoning proceeds fluently from a misread input, producing confident wrong answers.
Note (2026-11-06): the abstract of arXiv:2603.18472 (now v2; first and current abstracts share about 25 percent of their wording) has changed substantially since its first version. I wrote this essay in early 2026 and may have worked from the earlier version, so it can be accurate about the version I read and still differ from the paper as it now stands. I have not compared this essay with either version in this pass.