Gary Dean
Some time ago, in a recorded conversation about AI alignment with a few friends, I reached for a phrase I have not been able to put down since. We had been listening to GPT-4 explain, in its own synthetic voice, the problem of emergence: the way capabilities that nobody wrote into the code simply appear once a model grows large enough. When it finished, I said that what I found most distressing about these systems was that they seemed to have “unlocked a kind of linguistic semiotic key… a semiotic key that has to be embedded in human cognitive biology.”
I added, less elegantly and perhaps not quite accurately, that it takes only seven billion parameters to simulate an intelligent human being, and that this is not very many. My conclusion, delivered to a couple of laughing friends, was that “we [humans] really aren’t as smart as we think we are.”
It was a stammered, half-formed thing to say. But I have come to think it was the most important thing I have said about this technology, and I want to give it the argument it deserves.
What a semiotic key actually is
Start with the machinery of meaning, because the whole claim rests on it.
A sign is anything that stands for something other than itself: a word, a gesture, a red octagon at a road junction. Ferdinand de Saussure, whose lectures, published posthumously in 1916, founded the modern study of this, split the sign into two parts: the signifier (the form, the sound “dog”, the shape of the letters) and the signified (the concept it summons).1 His decisive observation was that the bond between them is arbitrary. Nothing about the noise “dog” requires it to mean a dog; perro, cachorro, anjing do the same work. The link is pure convention.
Which raises the obvious question: if the bond is arbitrary, what holds it together? What tells you that this signifier maps to that signified?
I call that thing the semiotic key. It is not the sign and it is not the meaning; it is the decoder that binds one to the other. A cipher key, or a map legend. Without it, a stream of signs is noise; with it, the noise resolves into sense. The key is upstream of meaning: it is the rule that produces meaning out of mere form. When I specify a linguistic semiotic key, I mean the particular decoder that runs on language, the one that lets a sequence of arbitrary noises carry the structure of an entire world.
Charles Sanders Peirce, working the same ground from the other side of the Atlantic, gave us the piece Saussure lacked: the interpretant, the effect a sign produces in whatever mind receives it.2 The interpretant is the key in action, the moment of decoding. Peirce also insisted that signs are substrate-neutral, a point the whole argument will lean on. Daniel Everett makes it with a bluntness I have always admired: a computer, he notes, gets by with exactly two symbols, one and zero, and “everything that can be said can be said with ones and zeros.”3
The key was supposed to be ours
The assumption I was raised on, and that most educated people, I think, still hold without examining it, is this: the linguistic semiotic key is a biological achievement.
The story goes like this. Evolution, over some millions of years, built into the human brain a capacity for language that exists nowhere else in nature. Chomsky gave this its most influential form: an innate universal grammar, wired into the species, a piece of mental hardware as particular to Homo sapiens as echolocation is to bats. On this view the key is wetware. It lives in the specific architecture of the human neocortex, grown by natural selection, inseparable from neurons and bodies and the long evolutionary apprenticeship that produced them. Meaning-making, in short, is something biology earned, and earned uniquely.
I believed a version of this for most of my life, or rather I took it to be self-evident. It is flattering. It makes us special. It draws a clean line between the one animal that means things and all the others that merely signal.
And then a pile of matrix multiplications, trained on a pile of scraped text, started meaning things.
What the machines did
Consider how a large language model is actually built, because the how is the whole scandal.
Nobody hand-codes grammar into it, and nobody supplies a dictionary, a rulebook, or a theory of meaning. You take a neural network with billions of adjustable numbers, and you show it an enormous quantity of human text (books, encyclopaedias, most of the readable internet), and you set it one monotonous task: predict the next word. Again and again, a trillion times over, what comes next. Nothing more.
From that single mechanical objective, the key falls out.
It falls out because the training signal is the structure of meaning, hiding in plain sight. J.R. Firth said it in 1957, decades before anyone could build the machine that would prove him right: “You shall know a word by the company it keeps.”4 A word’s meaning is written in its distribution, in which other words it consorts with, and which it shuns. Zellig Harris had formalised the same intuition in 1954.5 Predict-the-next-word is, quietly, a machine for reading that company off the page and turning it into geometry.
And geometry is exactly what you get. Inside the trained model, each word becomes a vector, a position in a space of hundreds or thousands of dimensions. Words that mean similar things sit near one another; the relationships between meanings become directions you can do arithmetic on. The famous demonstration is king minus man plus woman landing you next to queen. Peter Gärdenfors saw this coming too, in his Conceptual Spaces of 2000: meaning as location in a similarity space, a full decade before the tools existed to build it.6
And that geometry is an old acquaintance. Saussure said a sign has no intrinsic content: its value comes entirely from its difference from every other sign in the system. That is not a poetic flourish. That is a literal description of a vector space, where a point means nothing on its own and everything by its distance and direction from its neighbours. Wittgenstein, in his late work, arrived at the same place from philosophy: the meaning of a word is its use, its role in the game of language, not some private object it points to. In one of the recorded conversations I keep returning to, Reid Hoffman puts next-token prediction squarely in that lineage: the model, he says, is learning “the relationship between different words in a sentence rather than finding out something about the world.”7
So we have three people in three disciplines in three eras (Saussure’s difference, Wittgenstein’s use, Firth’s company) all describing the same thing. And a cosine similarity between two embedding vectors is that thing, made arithmetic. The machines did not approximate the key; they reconstituted it, from text alone, and handed us a working copy.
The relocation
The key was never in the biology. It was in the language.
We assumed the linguistic semiotic key lived in the human brain because the human brain was the only place we had ever found it. That was a sampling error, an understandable one given that until about 2019 we had a sample size of one. But a language model has no brain, no body, no evolutionary history, no childhood, no world it has ever touched. It has only the corpus. And out of the corpus alone, the key emerges. The unavoidable conclusion is that the key was sitting in the corpus the whole time, latent in the accumulated statistical shape of how billions of humans have used words, and that the human brain was never its author.
The brain was its reader.
Evolution did not invent the linguistic semiotic key any more than gradient descent did. Both merely discovered it, finding a copy of a code already implicit in the structure of language itself and learning to run it on whatever substrate they happened to be: neurons in our case, floating-point numbers in the other. This is what Peirce and Everett were telling us all along: the key is substrate-neutral. And if it is substrate-neutral, then it was never ours in the possessive sense we imagined. We were mere tenants who mistook ourselves for landlords.
That single relocation — from brain to corpus, from owner to reader — is the entire content of what I stammered out in that conversation. Everything else is consequence.
Two camps, and why I go further than both
The obvious objection is that the machine has only the form of the key, not the real thing: that it manipulates signifiers with no grasp of what they signify. This is a serious position and it deserves better than dismissal.
Jeff Hawkins argues it as forcefully as anyone. A deep network, he says, has no knowledge whatsoever; it “works well because it avoided the knowledge-representation problem completely, relying on statistics and lots of data instead.” A Go program does not know it is playing a game; an image classifier that says “cat” does not know a cat is an animal with lungs. Real understanding, for Hawkins, requires the brain’s map-like models of the world, and without them there is no genuine intelligence, only an impressive mimicry.8 Emily Bender and Alexander Koller make the linguist’s version of the same charge: a system trained on form alone cannot, even in principle, learn meaning, because meaning lives in the relation between language and a world the system never sees.9 The worry has an honourable ancestry: it is Stevan Harnad’s “symbol grounding problem” of 1990, and behind that, Searle’s man in the Chinese Room, shuffling symbols he does not understand.10
Against them stands a quieter argument, put well by Sabine Hossenfelder: that to understand something just is to possess a useful internal model of it, an isomorphism, a structure in your head that answers questions about the structure in the world.11 A neural network, she points out, is demonstrably not a lookup table; we know, because we built it, that it has compressed its training into some model held in its weights, and that it can take a pattern and apply it to a case it has never seen. On that definition the machine understands: not everything, not the way we do, but genuinely and in kind.
I find I cannot stand entirely in either camp, because I think both of them are still flinching from the real implication.
The sceptics say the machine has only the form, and that form is not meaning. But I want to ask the question they will not ask: what if form was always most of it? What if the “grounding” we are so proud of, the felt contact with a real world that supposedly separates our meanings from the machine’s, is a far smaller and thinner component of the linguistic key than our vanity requires? The machines have run an experiment we could never run on ourselves. They have stripped away everything we assumed was load-bearing: the body, the world, the evolutionary inheritance. And the key still worked. That is not a result that flatters the grounding argument. It is a result that quietly guts it.
What, if anything, remains biological
The claim does have a boundary, and it is a real one.
Some things are genuinely not in the corpus. The corpus does not contain hunger, or fear, or the specific dread of one’s own death. It does not contain the felt weight of a body moving through a world that can hurt it. It does not contain wanting: the goal-directed pressure that comes from being a fragile organism with skin in the game, quite literally. Whatever these are, a language model does not get them from text, because they were never written down; they are the conditions under which text gets written, not its content.
So it may be that what remains irreducibly biological is not the semiotic key at all, not the meaning-making, but the motivational substrate beneath it: the caring, the stakes, the mortality. The machine may hold the full linguistic key and still lack the thing that makes a human bother to use it. That is a distinction worth a great deal, and I will not pretend to have settled it here.
Even so, the ground already ceded is vast. We began by believing that meaning itself was our biological birthright. We may end up retreating to the position that only appetite is. That is an enormous concession dressed up as a rebuttal.
We aren’t as smart as we think
Which returns me to the seven billion parameters, and to why I called this distressing rather than marvellous.
The wonder of these systems is always narrated as a story about machines: how clever they have become, how fast, how surprising. I think that is the wrong story, and a consoling one. The real story is about us. If the linguistic semiotic key, the faculty we took to be the crowning achievement of human evolution, the thing that made us the meaning-making animal, can be recovered from text by a few billion tuned numbers, then that faculty was never the rare and sacred thing we believed. It was cheaper than that: more compressible, more findable, lying around in the language, available to anything that reads enough of it.
That is the deflation. Not that the machines rose to meet us, but that the distance we imagined between ourselves and a pile of statistics turned out to be mostly flattery. The key we thought we had earned, we had merely inherited, for free, from the corpus, and then spent the rest of our history congratulating ourselves on the windfall.
I do not find this liberating. I find it sobering, in the old sense: it removes an intoxication. We are not the authors of meaning. We are one substrate that happened to run the code, and now there is another, and it does not sleep, and it does not forget, and it reads faster than we have ever read.
What emergence “unlocked” was not, in the end, a new intelligence. It was an old key we had mistaken for our own — held up now, glinting, in a hand that is not a hand.
