Does the difference between statistical and semantic understanding matter?
On the philosophical stakes of how we characterise large language models
Does the difference between statistical and semantic understanding matter? This question has become somewhat commonplace in the discourse surrounding large language models. Critics insist that beneath the fluent prose these systems generate is little more than a sophisticated pattern matching engine that predicts the next token with incredible accuracy, but does so while remaining severed from any sense of meaning itself (and they’re not wrong). This distinction between statistics and semantics has hardened into a fault line separating skeptics from enthusiasts, raising fundamental questions about what it means to understand language and whether machines might ever do so. But the boundary itself warrants scrutiny. And perhaps more importantly, does it even matter?
The critique is crystallised most sharply in arguments advanced by computational linguists like Emily Bender and Alexander Koller, who contend that language models trained exclusively on text can never acquire genuine meaning because they lack grounding, the tether between linguistic forms and communicative intent in the physical or social world. Without reference to external reality, such models manipulate symbols according to distributional statistics alone, producing what have been termed ‘stochastic parrots’, systems that mimic language without comprehending it. This view draws on a venerable philosophical tradition. John Searle’s Chinese Room argument long ago distinguished syntactic manipulation from semantic understanding, suggesting that rule-following symbol processing—even when indistinguishable from human behavior—does not entail meaning. For contemporary skeptics like cognitive scientist Gary Marcus, large language models exemplify this limitation, exceling at what he calls “statistical glue” but failing at genuine reasoning, compositionality, and causal modelling of the world. The consequence, in this view, is that belief in machine understanding constitutes one of the most profound illusions of our technological moment.
