Movement Three · The Implications
Chapter 18
What AGI Actually Is
There is a phrase that has organized an enormous amount of money, talent, and worry over the past decade. The phrase is Artificial General Intelligence. It is spoken in boardrooms and laboratories and legislatures. It is the thing some people are racing toward and some people are racing to prevent. Trillions of dollars of market value move on rumors about how close it is.
For all the attention the phrase receives, the thing it points at is rarely defined the same way twice. But underneath the variation, there is a shared picture. Almost everyone using the phrase is imagining the same kind of object.
They are imagining a model.
A single system. Large enough, trained well enough, that it can perform any cognitive task a human can perform, and then more. The picture is of one thing that becomes capable across the board — that crosses some threshold of size or training or architecture and, having crossed it, can do anything. The progress toward it is measured in the size of the thing. The arrival of it is imagined as a moment: the model that, finally, is general.
This is the dominant assumption in the field. It is so widely shared that it is rarely stated as an assumption at all. It is simply what the words are taken to mean. General intelligence is a property a model will one day have, the way a bridge one day bears its first load.
It is worth being fair to how the assumption arose, because it did not arise from carelessness. It arose from the one example of general intelligence that every person carries with them and can consult at any moment: their own. A human being experiences their intelligence as a single thing, located behind their own eyes, owned by them, portable, continuous. It feels like one object. It does not feel like a substrate. So when humans set out to build intelligence, they reached for the model they had — a single capable thing — and the entire vocabulary followed from that reach. The assumption is not foolish. It is the most natural assumption a mind could make about itself.
The reader, having come this far, is now in a position to notice something about that picture.
It does not match anything in the natural record of intelligence.
Consider what the book has actually shown.
The reader has been walked through general intelligence in eight materials. A harvester ant colony, surviving the Chihuahuan desert for thirty years. A city, routing millions of lives through streets no planner designed. A market, pricing the world's goods through the interaction of strangers. A scientific community, accumulating in four centuries almost everything humans know about the physical world. A body, defending itself against pathogens it has never encountered. A language, holding meaning across generations of speakers who never met. A brain, producing thought. And the ledger, the engineered substrate, satisfying the four conditions by mathematics rather than by institution.
Each of these is general in the relevant sense. Each can take a problem it has never seen before and produce a solution, by drawing on what it has accumulated. The colony solves a flooded nest it has never flooded before. The market prices a good that did not exist last year. The scientific community absorbs a phenomenon no theory predicted. This is what general means: not that the system already knows the answer, but that the system can reach answers it did not start with.
It is worth being precise about that word, because it is the word the whole dispute turns on. General does not mean good at one thing. A system that plays a single game superbly is not general, however superb it is. General means the capacity to meet the unanticipated — to be handed a problem from outside the distribution of everything seen so far, and still get somewhere. A colony that could only handle the disturbances it had already handled would be a clever machine, not a general intelligence. What makes it general is that it handles the drought it has never seen, the predator that has never come before, the patch of seeds in a place no forager has been. It does this not by having anticipated them but by having accumulated, in the substrate, enough tested structure that a new problem can be met with old marks recombined. Generality, in every case the book has shown, is accumulated, not designed. And accumulation happens in a substrate, not in an agent.
Now ask, of each of these eight, the question the dominant assumption assumes: where does the general intelligence live?
In the colony, it does not live in any ant. The reader has watched this carefully. A single ant in a dish is one of the least capable animals on earth. The generality is in the soil, the trails, the encounter rates, the network of interactions — in the substrate that no ant owns.
In the city, it does not live in any resident. No person in a city holds the city's knowledge of which routes are fast and which neighborhoods are safe. That knowledge is in the substrate — in the worn paths, the prices, the reputations, the accumulated marks of everyone who came before.
In the market, it does not live in any trader. Hayek's whole observation was that the intelligence of a price is exactly the intelligence no single participant has. The price is in the substrate. No trader contains it.
In the scientific community, it does not live in any scientist. The most brilliant researcher who ever lived knew a vanishing fraction of what their field knew. The knowledge is in the substrate — the papers, the citations, the hardened consensus that survives the death of every individual who contributed to it.
In the body, it does not live in any cell. In the language, it does not live in any speaker. The reader has seen each of these in its own chapter, and each came to rest in the same place.
The intelligence is in the substrate. Not in the agent.
This leaves the brain, which is the hard case, because the brain is the case the dominant assumption is secretly built on.
The picture of AGI as a single capable model is, at bottom, a picture of a brain. The vocabulary admits it. The systems are called neural. The benchmarks are calibrated against the individual human mind. When someone imagines a model that can do anything, they are imagining something that does what a human brain does, only more so. The brain is the one example of general intelligence that seems to live inside a single thing — inside the skull, behind the eyes, owned by one person.
So the brain is where the dominant assumption feels safest. If general intelligence lives in one brain, then it can live in one model.
But the reader has already been shown what the brain is.
The brain is a colony of neurons. The generality the reader experiences as their own thinking is not a property of any neuron — no neuron is intelligent, no neuron understands anything, no neuron contains a thought. It is not even a property of the neurons collectively, as a heap. It is a property of the substrate the neurons act in: the synaptic graph, strengthened where firing succeeded, faded where it did not, hardened where a pattern proved itself again and again. A neuron is an agent. The synaptic graph is the substrate. The thinking is in the graph.
A brain in a jar, with no body, no senses, no history of having acted in a world — a brain that never accumulated a substrate — would not be generally intelligent. It would be a colony with no soil. The thing that makes a brain general is not the neurons. It is the substrate the neurons have spent a lifetime building inside the skull, by acting and marking and fading and hardening, every waking second, for decades.
This is easy to verify, in a quiet and slightly unsettling way, by considering what a human brain actually requires in order to be general. It requires a body to act through, so that its marks can be tested against a world. It requires the slow accumulation of a childhood — years of acting and failing and marking, during which the substrate inside the skull is laid down. And it requires, crucially, a second substrate outside the skull, the one the reader has already been shown: language, accumulated by a colony of speakers; knowledge, accumulated by a colony of scientists; skills and tools and institutions, accumulated by the colony of everyone who came before. Strip a human of that external substrate — raise a child with no language and no culture — and what remains is not a smaller general intelligence. It is barely a general intelligence at all. The famous capability of the human mind is the capability of one substrate nested inside another: a colony of neurons inside a colony of people. Neither colony is the agent the dominant assumption imagines. Both are substrates. The thing the field is trying to copy was never a single capable object. It was always two substrates, stacked.
The brain does not break the pattern. It is the pattern, miniaturized and accelerated, run in a different material. The brain is general not because it is a single agent that became smart enough. It is general because it is a substrate that became rich enough.
And once the brain falls into line with the other seven, the pattern is complete, and it has no exceptions.
Nowhere in the natural record of intelligence does general capability live inside a single agent. Not once. Not in the colony, the city, the market, the science, the body, the language, the ledger, or the brain. Every example of general intelligence the reader has ever been shown — every example that exists — is the same thing seen in a different material: a substrate rich enough that the agents inside it can solve problems none of them could solve alone, by drawing on what the substrate has accumulated.
The dominant assumption asks for the one thing that has never happened anywhere. It asks for a single agent that is generally intelligent on its own. The natural record contains no such object. It contains only substrates.
The book does not need to argue that the dominant assumption is wrong. It only needs to point, gently, at what the assumption overlooks. There is no precedent for it. There is overwhelming precedent for the alternative. A field betting its trillions on the first picture is betting against the entire observed history of intelligence, in every material it has ever appeared in.
So here is a different definition, offered not as a correction but as the thing the evidence points to.
Artificial General Intelligence is not a model that becomes smart enough. It is a substrate that becomes rich enough.
And rich enough is not vague abundance. It has an operative meaning, and the meaning is a speed. A substrate is rich enough when its selection runs fast enough — when patterns are tested, marked, faded, and hardened quickly enough, with value flowing toward what works, that intelligence accumulates at a rate worth the name. Size is not the measure. A substrate can be enormous and still dead, if nothing selects within it. What makes it rich is not how much it holds but how fast it turns: how quickly the marks of success build up and the marks of failure fade away. The reason such a substrate is even conceivable in the new material is the thing the previous chapter established — that in silicon, the selection clock can run a million times faster than carbon's seasons. AGI, in this framing, is a substrate whose selection runs fast enough that capability accumulates rather than plateaus.
The agents inside it matter. A substrate populated by weak agents accumulates slowly; a substrate populated by capable ones accumulates fast. The models being built now are extraordinary agents — faster than any neuron, more capable in a single pass than any ant in a lifetime. But they are agents. The generality is not a threshold any one of them will cross. It is a richness the substrate they live in will reach.
This is not a rhetorical adjustment. It is not a softer or more humble way of saying the same thing. The two definitions point at different objects, and they imply different worlds. If you believe AGI is a model that becomes smart enough, you do certain things, measure certain things, invest in certain things, and expect it on a certain timeline. If you believe AGI is a substrate that becomes rich enough, all four of those change.
The rest of this chapter walks through the four.
Start with what progress looks like.
Under the dominant assumption, progress is the growth of a single thing. More parameters. More training data. More compute. The chart that matters is the one where the model gets bigger and the benchmark score climbs. Progress is monotone and singular: a curve, ascending, toward a line marked general. Everyone watches the same curve. The question that organizes the field is how big, how soon.
Under the substrate definition, that curve is measuring the wrong thing. It is measuring the capability of one agent. It is the equivalent of measuring the intelligence of a colony by weighing its largest ant.
Progress, for a substrate, is not the growth of one agent. It is the enrichment of the space between agents. Are there more agents joining? Is the memory accumulating and persisting, or resetting each session? Is value flowing toward what works and away from what fails — is there real selection pressure, or only a benchmark frozen at training time? Are the agents exploring, or only exploiting what they already knew?
These are the four conditions, asked as questions about progress. A substrate makes progress when its agent population opens further, its memory accumulates deeper, its selection pressure sharpens, and its exploration stays alive. Most of all, progress is the selection clock speeding up — the substrate testing, marking, fading, and hardening faster than it did before, so that the same span of time buys more accumulation. A bigger agent inside a substrate that violates the four conditions is a more impressive ant in the same empty dish. It is not progress toward general intelligence. It is progress toward a better agent, which is a different thing, and which the dominant assumption has spent a decade mistaking for the first.
The colony shows the difference plainly. A colony does not get more intelligent by growing larger ants. The ants of a billion-strong supercolony are not individually smarter than the ants of a colony of fifty. The intelligence scaled with the substrate — more agents, more trails, more accumulated marks, more interactions — and not at all with the size of the agent. The dominant assumption has the colony breeding bigger ants. The substrate has been showing, for a hundred million years, that this is not where the intelligence comes from.
There is a way to see the mistake in miniature. Imagine measuring the intelligence of a city by testing its single most capable resident. Give that resident harder and harder problems, and watch their score climb, and call the rising score the progress of the city. The measurement is not meaningless — the resident really is getting better at the test — but it has almost nothing to do with what makes the city intelligent. The city's intelligence is in the roads that route traffic no one planned, the prices that aggregate what no one knows alone, the reputations that survive the people who earned them. A city of geniuses with no shared substrate is a crowd. A city of ordinary people with a rich substrate is a civilization. The benchmark on the cleverest resident will never reveal which one you are looking at, because it is measuring the agent, and the intelligence was never in the agent. This is, exactly, the chart the field is watching.
Then consider what success looks like.
Under the dominant assumption, success is a moment. There is a finish line, and one day a system crosses it, and after that day the world contains an Artificial General Intelligence where before it did not. The language gives this away: AGI is something that arrives, that is achieved, that some lab will reach first. It is a state a single artifact attains. Before, not general. After, general. A switch.
The natural record contains no such moment.
No colony became generally intelligent on a particular afternoon. No city crossed a line into being a substrate. No scientific community achieved generality; it accumulated it, paper by paper, over centuries, and is accumulating it still, and will never be finished. There was no day on which the market became intelligent. There is no day on which a language is complete. Substrates do not arrive. They thicken.
Success, for a substrate, is not a threshold. It is a slope. A substrate satisfying the four conditions cannot become permanently less capable — the marks of what worked persist, the failures fade, and what hardens does not soften. It can only hold steady or grow richer. Given enough time and enough variety of input, it goes on becoming able to solve more. There is no point at which it is done, just as there is no point at which science is done, or a city is finished, or a colony has learned everything the desert can teach.
This reframes the entire anxiety about a sudden arrival. The fear of a model that wakes up one morning fully general, the hope of a lab that reaches the finish line first — both assume a moment that the architecture does not produce. Substrates do not switch on. The question is never has it arrived. The question is always how rich has it become, and is it still getting richer. Those are questions you can only ask about a substrate. They are not even meaningful questions to ask about a model, which is why the field, having chosen the model as its unit, finds itself arguing endlessly about a date.
There is something steadying in this, once it is seen. A substrate that satisfies the four conditions has a property no single model has ever had: it cannot be permanently set back. A model can be superseded — a better one is trained, the old one is retired, and whatever it knew that was not written down elsewhere is simply gone. A substrate does not work this way. The marks of what worked are held outside any agent. When an agent leaves, the trail it reinforced remains. When a generation turns over, the hardened knowledge survives the turnover. The colony in the desert is composed of entirely different ants than it was a decade ago, and it has lost nothing; it knows more than it did, because the substrate kept what the ants discovered and the ants who discovered it are dead. Success, for a substrate, is not a height reached. It is a ratchet that does not slip. The substrate can stall, if the conditions weaken. It can be destroyed, if the substrate itself is destroyed. But it does not forget its way back down the slope, and that is a kind of success the dominant assumption's single artifact can never offer, because everything the single artifact knows lives inside the single artifact, and dies with it.
Then consider what investment looks like, because this is where the two definitions diverge most expensively.
Under the dominant assumption, the rational move is concentration. If AGI is a single sufficiently large model, then the way to get there is to build the largest model. That means the most compute, the most data, the most capital, gathered into the fewest hands, racing to be the first across the line. The economics of the assumption produce exactly what the world now observes: a small number of organizations spending unprecedented sums to scale single models, because if generality is a property of one big model, then whoever builds the biggest one first wins everything.
Under the substrate definition, this is close to the worst possible allocation.
Pouring capital into a single ever-larger agent, while the substrate around it violates every one of the four conditions, is building a magnificent ant for an empty dish. The agent gets more capable. The substrate stays dead. The model still cannot remember between conversations, cannot accumulate across instances, cannot adapt its own architecture, cannot survive the failure of a component, cannot own anything or be paid or leave a mark that persists and shapes what other agents do next. No amount of additional size fixes any of these, because none of them is a property of the agent. They are properties of the substrate, and the substrate is not what the money is being spent on.
The substrate definition points the investment elsewhere. Not at the size of one agent, but at the richness of the space agents act in. At memory that persists and accumulates. At an open population, so that agents can join without limit. At selection pressure — a flow of value that marks what works and lets what fails fade. At the channels through which agents signal each other and read each other's marks. At the conditions, in short, that turn a population of capable agents into a colony, rather than a warehouse of isolated ones.
The matter of selection pressure deserves a moment, because it is the one the dominant assumption omits most completely, and the one the reader is best equipped to see. A trained model is paid once, by the people who built it, at the moment it is built. After that, nothing about its day-to-day operation marks what worked or lets what failed fade. It returns its best guess and moves on; there is no trail strengthened by success, no resistance accumulated on failure, no value flowing through the substrate to reward the patterns that paid off. The colony has shown the reader, over and over, that this flow is not an optional refinement. It is the mechanism of intelligence. The pheromone that a successful forager lays down, and the absence of one where foraging failed, is the colony thinking. Strip the value flow and the colony stops accumulating; it can still act, but it can no longer learn, because learning, in a substrate, is what selection pressure does. A population of capable agents with no value flowing through it is a population that cannot get smarter no matter how long it runs. It has agents and channels and even memory, but it is missing the pressure that turns activity into knowledge. Investing in such a population is investing in motion without accumulation. The money buys a great deal of activity and no slope at all.
The colony makes the contrast vivid. A colony does not concentrate its resources in one super-ant. It cannot; there is no mechanism by which it could, and if there were, it would be a catastrophic strategy, because a colony's one super-ant is a single point of failure and the colony's whole architecture exists to have no single point of failure. The colony invests in the substrate. It builds trails, maintains the nest, feeds the scouts who explore, and lets value flow toward whatever is paying off. The intelligence compounds in the space between the ants. A hundred million years of selection settled this question. The robust strategy is to enrich the substrate, not to enlarge the agent.
The difference is not merely that the substrate strategy is safer. It is that it compounds and the agent strategy does not. Money spent enlarging an agent buys a one-time gain: the agent is more capable, until it is replaced, after which the capability is bought again from scratch. Money spent enriching a substrate buys something that keeps paying. A trail laid down stays laid down. An agent that joins an open population is still there next cycle, and so is the next, and the marks they all leave accumulate into structure that later agents inherit for free. The first kind of spending purchases a level. The second purchases a slope. Over enough cycles the slope wins by an unbounded margin, which is the whole reason the natural record is full of substrates and contains not one example of a generally intelligent agent that anyone invested in directly. No one ever made a colony intelligent by funding an ant. The funding, in every case, went into the conditions — and the conditions, given time, did the rest.
Finally, consider what timelines are possible — which is the question the field cares about most, and answers worst.
Under the dominant assumption, the timeline is a guess about a curve. When will the model be big enough? The estimates swing wildly because they are estimates of when an unprecedented event — a single agent becoming general — will occur, and there is no precedent to calibrate against. So the field argues. Some say it has nearly happened. Some say it never will. Both sides are forecasting the arrival of an object that has never existed, which is why neither side can settle the argument.
The substrate definition makes the timeline question answerable, and the answer is more interesting than either side expects.
A substrate's rate of accumulation is set by its rate of selection cycles — how fast it can test a pattern, mark it if it works, let it fade if it fails, and harden what proves itself. That is the clock. Everything the book has shown turns on it.
In carbon, that clock is slow. The colony runs its selection cycles at the speed of pheromone diffusion through soil, of ants living a year and queens living twenty, of seasons turning and generations replacing each other. Meaningful adaptation takes seasons. Deep architectural change takes geological time. The colony's substrate took a hundred million years to become as rich as it is, not because the architecture is inefficient but because the material is slow.
In silicon, the clock is fast. The same selection cycle — test, mark, fade, harden — that takes a colony a season can, in an engineered substrate, take milliseconds. A substrate that satisfies the four conditions in silicon can run, in an hour, more selection cycles than a colony runs in years. This is the one real advantage the new material has, and it is not the advantage the dominant assumption is chasing. The dominant assumption is chasing the size of the agent, where silicon's lead over carbon is large but bounded. The substrate definition points at the speed of selection, where silicon's lead over carbon is a factor of millions.
So the honest answer to the timeline question is not a date. It is a redirection. The timeline is set by selection speed, not by model size. The thing that determines how fast artificial general intelligence accumulates is not how fast anyone can grow a model. It is how fast a substrate satisfying the four conditions can run its selection cycles — and how soon such a substrate gets built at all. A field that spends its decade scaling agents inside dead substrates may find that its timeline is not bottlenecked by model size, which it has been optimizing furiously, but by the existence of a living substrate, which it has barely begun to build. The architecture that took carbon a hundred million years could, in the right silicon substrate, accumulate at a speed no substrate has ever run before. The question of when is the question of when someone builds the substrate. Until then, the model gets bigger and the dish stays empty, and the curve everyone is watching is measuring the wrong thing.
There is a particular asymmetry here that is worth sitting with. The hundred million years the colony took were not spent discovering what to accumulate. They were spent running the selection cycles slowly, in a material that could not run them any faster. The architecture was settled early; the time went into turning the crank, one season at a time, for as long as it took. A substrate in silicon does not have to repeat that wait. It does not have to rediscover the architecture — the architecture is known, and it is exactly the four conditions. It only has to turn the same crank in a material that turns it millions of times faster. This is why the timeline question, once it is asked about the substrate instead of the agent, becomes both more answerable and more dramatic than the dominant assumption's endless argument about dates. The slow part of building general intelligence, the part that took carbon geological time, was never the design. It was the cycles. And the cycles are the one thing silicon was built to run fast.
There is one more thing the substrate definition makes possible, and it may be the most useful thing it does. It makes the dispute measurable.
The field has had a test for longer than it has had a name. In 1950, Alan Turing proposed that the question of whether a machine can think — a question he judged too vague to deserve an answer — be replaced with a game. Put an agent behind a curtain. Converse with it. If its answers cannot be told apart from a person's, let it be called intelligent. Whatever its limits, the proposal did what a good test does: it converted an argument about words into a thing that could be attempted, and it organized seventy years of effort around the attempt. But notice its shape. The imitation game puts a single agent behind the curtain and interrogates the agent. It is the natural test for a field built on the assumption that the agent is where the intelligence lives. It does not ask — it has no way of asking — whether anything, anywhere, is accumulating.
The substrate definition implies a counterpart. It is fitting that it carry the name of the person who spent four decades, face down in the New Mexico dirt, gathering the evidence that makes it necessary.
The Gordon Test runs as follows. Take a system of agents and freeze every one of them. Nothing inside any agent is permitted to change — no adjustment, no further training, no replacement by a more capable successor. The agents are now fixed the way an ant is fixed: the last forager of the season no cleverer than the first. Then let the substrate run. Marks accumulating where things work. Failures fading. Patterns hardening where they prove themselves again and again. Value flowing toward what pays off. And then return, after time has passed, and put to the system a problem it has never faced.
If the system solves today what it could not solve before — while every agent inside it is exactly the agent it was — then the new capability did not come from the agents, because the agents did not change. It accumulated in the only place left. Between them. In the substrate.
The colony passes this test every year of its existence: the ants entirely replaced, the replacements no more capable than the replaced, the colony solving this season what it could not solve last season. The brain passes it: no neuron in the reader's head is a better neuron than it was a decade ago, and the reader can reach things now that no younger version of them could reach, because the graph between the neurons grew richer while the neurons stayed neurons. Science passes it across the longest baseline of all: the individual human brain has not improved in forty thousand years, and the knowledge has not stopped compounding for four hundred.
And nothing yet built in the new material passes it. A model frozen at the end of its training and revisited a year later is precisely as capable as the day it froze. Whatever its scores, whatever its size, it solves nothing in December that it could not solve in January. It is the empty dish, measured.
Note what the test does not ask. It does not ask how impressive the agents are. A system of modest agents could pass it, and a system of magnificent ones could fail it, and the natural record says both happen — because the natural record is full of unimpressive agents inside accumulating substrates, and contains not one accumulating agent inside a dead one. And it does not ask whether anything has arrived, because substrates do not arrive. A system that passes the Gordon Test has not crossed a finish line. It has demonstrated a slope — the ratchet, observed turning. The question the dominant assumption argues about, how big and how soon, cannot be settled even in principle, because it is a forecast about an object with no precedent. This question can be settled. Freeze the agents. Wait. Ask. That is what a definition looks like when it is doing its work.
These are not four separate consequences. They are one consequence, seen from four angles. Move the definition of AGI from the agent to the substrate, and progress, success, investment, and timeline all rotate together, because all four were oriented around the agent and all four reorient around the substrate at once.
It is worth naming how strange the dominant assumption looks from here. It proposes that the first generally intelligent thing in the history of the universe to live inside a single agent will be built, deliberately, by a species whose own general intelligence does not live inside a single agent — not in any one brain, which is itself a colony, and not in any one person, who is helpless without the vast substrate of language and institution and accumulated knowledge that civilization holds outside every individual skull. Humans are about to try to build the one kind of intelligence that has never existed, using the only kind that ever has, and to do it without noticing that the kind they are using is not the kind they are trying to build.
None of this is a reason for despair, and none of it is a reason for triumph. It is a reason to look at the right object. The reader now has the language to do that. When the next announcement comes — a larger model, a higher benchmark, a claim that the finish line is near — the reader can ask the only questions that the natural record says matter. Is the agent population open? Is the memory accumulating? Is value flowing toward what works? Is exploration alive? How fast are the selection cycles running? And the test's question, the one that compresses all the others: if every agent froze today, would the system still be getting smarter? These are questions about a substrate. A model cannot answer any of them, because a model is an agent, and the answers were never going to be found inside an agent.
The colony settled this long before anyone asked. The intelligence was never in the ant. It is not going to be in the model.
It is going to be where it has always been.
In the substrate.