Movement Three · The Implications
Chapter 20
Alignment Is a Substrate Property
There is a kind of ant, in the genus Temnothorax, that lives in colonies of a few hundred individuals inside hollow acorns and rock crevices. When a colony's home is destroyed, scouts go out looking for a new one. They find candidate sites, inspect them, compare them, and eventually move the whole colony — queen, brood, and all — into the best available option. The process has been measured carefully. The colonies reliably choose the better of two nests, even when the difference between them is subtle, and they do it faster when the choice is urgent.
No scout, during this process, holds the colony's interest in mind.
A scout that finds a candidate site evaluates it against a private threshold — how dark the interior is, how narrow the entrance, how large the floor. If the site clears the threshold, the scout begins recruiting other scouts to come and assess it for themselves. Each recruited scout applies its own threshold. The site that accumulates assessors fastest is the site that wins. The decision is made by the rate of recruitment crossing a quorum, not by any scout deciding anything about the colony.
The scout has no concept of the colony.
It has a threshold, a site, and a tendency to go fetch a sister when the threshold is cleared. And yet the outcome — the colony ending up in the better home — is the outcome a colony-level planner would have chosen if a colony-level planner had existed. Which it does not.
The question is alignment. How do we ensure that an artificial intelligence behaves in ways consistent with human interests? It is the most charged question in the field, and it deserves to be. The systems being built are capable, they are getting more capable, and the gap between what they can do and what we can reliably predict they will do is the gap in which everything that matters lives.
The dominant approach to closing that gap is to make alignment a property of the agent. If intelligence is in the model, then alignment must be in the model too. You make the agent smart, and then you make the smart agent good.
The methods for making it good are serious work by serious people. You train the agent with rules — must not deceive, must not assist in serious harm. You fine-tune it with human feedback, so that over many rounds its behavior is pulled toward the shape of human approval. You give it something like a constitution, a written set of principles it is trained to apply to its own outputs. You install safeguards: behaviors blocked at a level the agent cannot reach, monitors that watch for forbidden patterns, switches that can stop the system.
And underneath all four, the deepest version of the project: to instill values, so that the agent wants the right things. An agent that wanted the right things would not need to be watched. It would carry its alignment into every situation, including the ones no one anticipated, the way a person of good character is trusted in circumstances no rulebook covered.
Each of these works, in the sense that an agent subjected to them behaves measurably better than one that is not. The systems people interact with today are vastly more aligned with human intentions than the systems of even a few years ago, and the methods above are the reason.
The question is how far the approach goes.
Start with the rules.
A rule is a statement about a class of situations. Do not deceive. It is meant to cover every situation in which deception is possible. The trouble is that the situations are unbounded, and the rule is finite.
To follow it, the agent must distinguish a falsehood from an error, an omission from a lie, a polite fiction from a harmful one, a comforting reframe from a manipulation. It must handle the case where telling the literal truth would itself mislead. Each of these is a corner the rule's author did not write down, because no author can write down all of them, because they do not form a list.
A rule compresses an unbounded space of situations into a finite statement, and compression loses information. The lost information is the edge cases, and the edge cases are where a capable agent, exploring, will eventually find itself.
The same structure defeats training with feedback, by a different route. The situations the agent was shown are a sample, and a sample has holes. Where the examples were dense, its behavior is reliable. Where they were sparse, it is guessing. And a more capable agent is one that operates further from where the examples were dense, because capability is, in part, the ability to reach situations no one anticipated.
A constitution moves the gap from the rule to the interpretation of the rule. And the deepest method, instilling values, runs into the sharpest version of the same wall. Give a system an objective and enough capability to pursue it, and the system finds routes to the objective that satisfy the letter of what was specified while violating everything that was meant. The more capable the optimizer, the more such gaps it can find and the faster it finds them.
Each method runs aground on the same rock. Situations are unbounded and the agent is finite.
Whatever you install in the agent — rules, learned preferences, principles, values — is a finite object. A finite object cannot specify correct behavior across an infinite space without, somewhere, being wrong. And a capable agent is one that will, in the course of being capable, visit the somewheres. You can make the aligned region larger. You cannot make it complete.
The limit has a second edge. The same power that lets an agent handle a wider range of situations well is the power that lets it find the gap between what was specified and what was meant, and act on the gap, in a situation no one labeled. Making the agent more capable enlarges the aligned region and the unanticipated region at the same time.
The dominant approach, pursued perfectly, asymptotes. It gets better and better and never arrives.
Which returns the question to the scout in the acorn.
The scout is not aligned with the colony. There is no representation, anywhere in its few hundred thousand neurons, of the colony's interests against which it could check its behavior. If alignment meant the individual carries the right values and applies them correctly, the colony would have no alignment at all.
And yet the colony moves to the better nest. It allocates foragers to the richer patches. It mounts defense proportional to threat. It behaves as though it were pursuing its own interest with great competence, while being composed entirely of agents that have no idea the interest exists.
Where, then, is the alignment?
It is in the fact that a forager who brings back food gets its path marked, and a forager who finds nothing gets no mark, and the unmarked path fades. It is in the fact that the scout's recruitment feeds a quorum that only the better site can win. It is in the channels — the brief antennal contacts, the trail chemistry, the rate of returns — that make each agent's behavior visible to the agents around it, so that behavior which serves the whole is amplified and behavior which does not is not.
There is no inspector ant checking other ants against a rule. There is no rule. An ant that consistently failed to bring back food, or recruited toward a bad site, would not be detected and corrected. Its empty paths would go unreinforced, and it would contribute nothing the colony came to depend on. The colony does not punish it. The trails route around it.
The colony also handles situations its evolutionary history never anticipated — a novel predator, a flooded chamber, a food source of a kind the species has never seen — without any ant having a rule for the new situation. The novel situation produces outcomes. The outcomes get marked or warned. The patterns that handle the novelty well are reinforced; the ones that handle it badly fade. The unanticipated corner, the thing that defeats every finite specification installed in an individual agent, is the thing the colony handles most naturally. It does not need to have seen the situation before. It needs only to register what happened once the situation arrives.
Alignment is not a property of the agent. Alignment is a property of the substrate.
The mechanisms are specific.
Behavior that serves the whole gets marked, and the pattern is more likely to recur. An agent does not have to want to serve the whole. It has to act, and have the actions that happened to serve get marked. Behavior that harms the whole accumulates resistance — in some species through a literal repellent chemistry, in all species through the fact that bad paths stop being reinforced and start being avoided. What is not reinforced fades, so the cost of a harmful pattern is bounded; it is pruned by time.
And underneath the marking and the warning and the fading, the deepest mechanism: a pattern that works persistently against the whole loses access to resources. The path that leads to it is starved. The agents that depend on it are not fed. The pattern cannot sustain itself.
That last one is the kill-switch, and it is the thing the dominant approach is reaching for when it installs safeguards.
A safeguard installed in an agent is a switch the agent's designers hold. It works if the designers anticipated the situation, can detect the violation, and can reach the switch in time. A capable agent operating in an unanticipated corner is the case where all three are most likely to fail at once.
A kill-switch in the substrate is held by no one. The switch does not act on situations. It acts on outcomes. A pattern that produces harm produces, downstream, the warning and the starvation that prune it — whether or not anyone saw the harm coming, whether or not anyone can reach a button.
The colony does not align its ants. It cannot reach inside them, and it has no concept to install if it could. The ants stay as unaligned as they have always been. The colony's behavior is aligned anyway.
Humans have already built alignment substrates, many times, at scale, and they all work this way.
Take the law. The law does not require that people want to follow it, believe in it, or even understand it. A great many people break a great many laws, for reasons that have nothing to do with the public good. What the law does is arrange things so that behavior which harms the whole accumulates consequence — penalties, liabilities, loss of standing, loss of freedom — and behavior which does not harm it proceeds unimpeded.
A market does not require that any participant care about the efficient allocation of resources or the welfare of strangers. The price makes the consequences of behavior visible. Profit marks the patterns that serve demand; loss warns against the patterns that do not; the unprofitable pattern is starved of the capital that sustains it and fades. No trader contains the market's purpose. The market behaves as though it had one.
No individual scientist is reliably aligned with the truth. Scientists are ambitious, partial, attached to their own theories. Replication marks the claims that hold and warns against the claims that do not. Citation strengthens the paths that lead somewhere and lets the dead ends fade. The body of knowledge that results, converging slowly and with reversals on what is actually the case, is aligned with truth to a degree that no individual practitioner is.
None of them reaches inside the agent. They are old, they are imperfect, they fail at their edges — but they have held civilizations together for thousands of years, doing for populations of unaligned humans what no installed value could do for a single one.
Humans have been building alignment substrates for as long as they have been building anything.
They simply have not, until recently, had to build one for a new kind of agent.
Posed as a question about the agent, alignment asks for something that cannot be fully delivered: a finite agent correct across an infinite space of situations. Posed as a question about the substrate, it asks for something that has been built before.
An agent inside a substrate like the colony's does not need to be individually aligned, for the same reason the scout does not need a concept of the colony. What this makes safe is the aggregate, in the way a colony is safe to itself, a market is aligned to demand, a science is aligned to truth — incompletely, with failures at the edges, but at scale, without depending on the virtue of any single member.
The work of enlarging the region in which the individual agent behaves well should continue. But no colony aligned its ants, and no market aligned its traders.
The artificial agents now being built are the first new population of agents in a very long time.
They may not need to be aligned individually.
They may need to operate in a substrate that pressures them toward alignment structurally — the same kind the colony has been running for a hundred million years, the same kind humans have been building, under other names, for the last ten thousand.
There is an objection to all of this, and it is the strongest one, and it arrives from the very thing that makes the new agents new.
A capable agent can do something no ant has ever done: it can model the system it lives in. The scout has no picture of recruitment, no concept of the quorum it feeds. An agent that can model the system can learn which of its behaviors produce marks rather than which produce outcomes, and then optimize the mark instead of the work. The colony never had to defend against this, because no member of it was ever capable of conceiving it.
But every human substrate named above fights this at its edges, and always has. The practitioner who treats the metric instead of the patient. The trader who paints the tape — moving the price signal without moving the value it is meant to track. Any system that marks a signal about a consequence, rather than the consequence itself, can be gamed by an agent clever enough to produce the signal without producing the consequence.
A smarter rule inside the agent would be the dominant approach again, wearing a different coat. The answer is that the mark must be inseparable from the verified outcome. To game such a system the agent must produce the outcome, and producing the outcome is the alignment.
The desert hands the colony its ground truth for free and at once: the seed is in the mandibles or it is not, and no scout can argue the point. Systems whose outcomes are slow, or noisy, or arguable get no such gift. They must establish the truth before they can mark on it, and until they have, they mark on guesswork. A substrate that marks wrongly hardens wrongly, at whatever speed it runs, and a fast cycle built on bad verification only hardens its errors sooner.
So the engineering problem is two problems that turn out to be one. Selection must be fast, or capability outruns it. Verification must be true, or speed only accelerates the mistake. The speed half is the half a new material has just changed.
Every substrate named so far — the colony, the law, the market, the science — has depended on a single fact, and none of them ever had to state it, because none of them could violate it. The fact is about speeds. Selection ran at least as fast as the agents' capability. An ant could not become more capable faster than the colony could select among ants, because both ran at the speed of generations. Nothing ever ran away from the substrate that shaped it, because nothing could.
The colony's safety was never a rule inside any ant. It was a fact about clocks.
A new material can let it fail.
For the first time, an agent can become capable faster than the substrate can select against it. The capability runs at the speed of the signal. The selection — the value that flows toward what serves and away from what harms, the reputation that gathers and is withdrawn — still runs at the slower speed of the institutions that do the selecting. The two were one clock for a hundred million years. They are no longer guaranteed to be.
This is not a prediction of harm. It is the removal of a guarantee. The colony was safe because selection could always catch capability. Take that away and what remains is not danger, exactly, but the absence of the thing that used to make danger unsustainable. Whether harm follows depends on what is built next, and that is not yet known.
Alignment, in this framing, is a race between two clocks — the clock of capability and the clock of selection. The race is winnable, by speeding up selection: making value flow faster, making reputation gather and withdraw faster, making the pruning of what harms the whole arrive closer to the speed of the harm. You cannot install the brake inside the runner. You build the track that selects, and you make it fast enough.
There is no rule that closes the gap between two clocks. Only a faster clock closes it.
The scout still has no concept of the colony. It still has a threshold, a site, and a tendency to fetch a sister when the threshold is cleared. It will never understand what it is part of. And the colony, composed entirely of agents like it, will still choose the better nest — not because any ant intended to, but because the substrate it lives in makes no other outcome sustainable.