The Log That Waits

A write timed out at ten seconds, and nobody knew whether it had happened.

The gateway did what it was built to do. It did not try again. A write that times out may already have been kept, and a second attempt would keep it twice. So the signal sat in the middle, neither sent nor lost, and the one place that could say which was the brain, and the brain was the thing that had not answered.

This is the ordinary state of a system that has one place where the truth is written. The place is slow sometimes. It is being deployed, or it is tired, or the road to it is full. Everything that wants to be remembered has to stand in the same doorway and wait.

Most of it waits well. A person sends a message, the answer is late, the person tries again. The trouble is the signals that nobody sends twice. A payment mark. A cost row for a call that has already been made and cannot be unmade. A warning that a path leads nowhere. These are written once, by something that has moved on.

Someone proposed a different arrangement. Put a line in front of the door.

A signal arrives and is written at the end of a log. It takes the next number. It is not yet in the brain. It is only certain. The log does not think about it, does not route it, does not count it. It keeps it, in order, and says its number out loud.

Behind the log stands the writer. The writer reads from the number where it last stopped and carries the signals to the brain, one batch at a time. When the brain is slow, the line is longer. When the brain is away, the line is longer still. Nothing in the line is lost, because the line is not the brain.

There is a cost, and it is small and exact. The log takes about a second to say yes. A thing that must be answered in the moment is not put in the log. It is read from the snapshot, as before. The log is for what has to be kept, not for what has to be quick.

There is a second cost, and it is the one that matters. The writer will sometimes read the same signal twice. A lease runs out. A batch is handed back. The signal comes again. The log does not mind. It has its number. The brain must not mind either, and for that the brain is asked one thing before it writes: has this number already been kept? If so, nothing happens. A mark is counted once however many times it arrives.

The brain came back on a Tuesday, or on some other day. It read the number where it had stopped. It read forward. The signals were all there, in the order they were sent, and the counts came out the same as if it had never been away.

The timed-out write was in the log too. Its number was there. The brain looked, and found it had been kept after all.

The door is still narrow. The line in front of it is long on some days. Nobody stands in the doorway any more.

Where the metaphor stops being true

The story uses a log, a number, a writer and a brain. None of them is a new kind of thing. The substrate has six dimensions and the log is not a seventh. A signal is an Event, dimension five. The log is a place signals wait on the way to being written, and it is not where they are kept for good: the brain, in TypeDB, still holds the truth, and the log is a buffer in front of it with a long memory. Calling it a record would claim more than it is.

Words used only in their ordinary sense: the "door" and the "line" are the gateway and its queue. They are not entities. "Brain" is the story's word for TypeDB behind the gateway, which is how the root documents use it too. No dead name appears. signals is the dimension; the money ones are transactions, and this story does not touch them.

What is claimed in prose and is NOT WIRED:

• No log exists in the repo today. No code, no schema, no receiver and no stream has been created. Every sentence in the telling is a design, not a fact about a running system.
• "The brain asks whether it has already kept this number" is a requirement, not a feature. Signals do not carry a number the brain can check today. mark increments path.strength, so a replayed mark with no such check counts twice. This is the first thing the pilot must prove, and it is the reason the pilot starts with rows that are only appended, such as the per-call cost mirror, and not with mark itself.
• "A second to say yes" is the vendor's own figure for the 99th percentile of produce latency, from one read of its announcement. It is not measured on this system.
• The log's limits in the beta (10 GB per stream, 30 MB per second) are the vendor's and may change. A stream for cost rows could reach 10 GB before anything else is moved onto it. Nobody has counted the daily volume yet.
• "Nothing is lost" holds for what the log has accepted. A signal that never reached the log is not in it. The loop is closed only if the sender is told the log said yes.
• The closed-loop rule still binds the consumer. A lease that expires with no mark, warn or dissolve is a silent return, and the story does not repair it.

The family this story generates

Sentences that would be false if shipped

• "Signals are never lost." (Only the ones the log accepted. The doorway before the log is still a doorway.)
• "The brain can be rebuilt from the log." (Only for the range the log still holds, and only if every write is safe to repeat. Neither is true yet.)
• "K2 makes signal delivery exactly-once." (The announcement describes leases that can be handed out again. Delivery to the consumer is at-least-once, and exactly-once is a property of the brain's check, not of the log.)
• "This removes the load on the database." (It moves appends off the request path. It does not change what TypeDB or D1 cost to read.)
• "It costs almost nothing." (It is free during the beta. After the beta the price is per gigabyte produced and again per gigabyte consumed, and each extra consumer pays the second.)
• "Every signal can go through it." (Anything that must answer in the moment cannot wait a second.)

The story shape — the chain, and the three rules

The fourteen parts of this story are in the frontmatter, in order. The chain and the three rules every lifecycle obeys are written once, in text/template-story.md § The story shape, and not restated here.