Every AI company will tell you their system learns. It’s in the deck, it’s on the landing page, it’s implied by the word “intelligence” doing unpaid overtime in the product name. Ours said it too. We even had the math written down, months before most of the code existed:
weight = 1 + max(0, strength − resistance) × sensitivity
One formula for all routing. Every time work goes well, the path to whoever did it gets stronger. Every failure adds resistance. Silence decays. The next decision reads the weights and gets a little smarter. Simple, honest, self-correcting.
This week we went looking for the learning, receipts in hand.
Here’s the part nobody wants to say out loud about systems that “learn”: in the entire life of ours, not one weight had ever moved.
The autopsy
Not a design flaw. The design was fine — the design was the best thing we own. What we found instead was three small failures stacked in a trench coat, each one silent, each one covering for the others.
First, a stale API key. The web layer had been authenticating to the database gateway with a credential that stopped matching at some point nobody can name. Every single write came back 401.
Second, a polite wrapper. The function making those calls caught every failure and returned an empty array, because years ago that seemed friendlier than crashing. So the 401s didn’t even have the decency to be errors. They were just … nothing.
Third, a dialect problem. Our database’s query language had moved under us, and the syntax we used for “increment this number” was no longer syntax. Creating a path worked. Strengthening one never did. The system could meet someone new; it could never learn to trust them.
A mirror copy of the weights in a second database kept updating the whole time, which is why the dashboards looked alive. The nervous system fired on every signal. It just never remembered anything.
We’ve all read the postmortems where the root cause is one dramatic thing. It’s never one dramatic thing. It’s a key, a catch block, and a semicolon, holding hands.
The first breath
Fixing it was a week of unglamorous work: re-key the gateway, rewrite every increment as read-then-write because the database refuses arithmetic in place, teach the failure paths to fail loudly, and then test the whole loop end to end in production, on purpose, with cleanup.
What came out the other side is the part worth writing down. We seeded a path at strength 5, routed two real signals through the live system, and read back 7. Routed more work: 8. Ran the daily decay tick against a path sitting at 10 and read back 9.5 — exactly the ×0.95 the formula promises. Then we poisoned a path deliberately, three heavy failures in a row, and watched the router do the thing we’d only ever claimed it would do: it stopped using that path entirely and escalated the next signal to a human.
Learn. Exploit. Explore. Forget. Refuse. Five behaviors of a living thing, all from one line of math, all verified against production with numbers we didn’t write ourselves.
There’s also now exactly one way in. You don’t tell our system who should handle something. You say it once — a signal, some tags — and the world decides: the proven handler if one exists, a promising newcomer sometimes (so early winners can’t squat on a topic forever), and a human desk if nobody’s listening. The address grammar for this was written into our routing spec months before the receiver existed. When we finally built it, the old grammar worked without a single line of glue. That’s the closest thing to a compliment a design ever gets.
The uncanny part
Midweek, an automated security review flagged a real hole: the learning weights were global, so one customer’s feedback could, in principle, poison another customer’s routing. Bad. Worth fixing immediately.
Two of our AI agent sessions were working in the repo at the time. Neither knew about the other. Both saw the finding.
One fixed it structurally — it scoped every learned weight to the workspace that earned it, so cross-tenant poisoning became impossible by shape rather than by rule. The other fixed it procedurally — it authenticated the write endpoints and blocked self-dealing, so an agent can’t inflate its own reputation. Different sessions, different instincts, zero coordination. The two fixes merged without a single conflict, and the union was stronger than either alone.
We started this company on a bet that intelligence comes from a colony of agents rather than from any single model. We did not expect the first clean demonstration to happen inside our own git history, on the codebase that implements the bet, before any customer saw it. The system building the system exhibited the property the system exists to produce. We’re still deciding how smug to be about this.
Where it actually stands
Calibration, because the alternative is becoming the landing page we opened by mocking: the organism is newborn. One door, a handful of callers, weights still sparse, a mirror ledger that will eventually need to be reconciled into one. A heart that has beaten is not the same as a circulatory system.
But the difference between this week and every week before it is not a feature. Before, every signal through the system left it exactly as smart as it found it. Now every routed signal moves a weight, every quiet day decays one, and every failure teaches the router what to refuse. 3,969 tests pass. The formula runs inside the database. The numbers moved and we watched them move.
Every AI company will tell you their system learns. Ask them when the weights last changed, and whether they watched.