Are We Drifting? — Part 8: Projected Truth
Are We Drifting? — Part 8: Projected Truth
Section titled “Are We Drifting? — Part 8: Projected Truth”Part 7: Models at Boundaries named events as the boundary that records facts. This part is about what you can build once facts are the source of truth — and why it is the most complete answer to drift we have found.
Are we drifting here?
Section titled “Are we drifting here?”Every useful screen is backed by a read model: a denormalized, query-shaped view of state. A kanban board. A list of active runs. A roll-up of which pull requests touch which tickets.
A read model is, by definition, a copy of truth arranged for fast reading. And the first law of this series is that copies drift. The board says a ticket is in review; the underlying state moved on; a consumer that updates the board missed an event, or processed one twice, or crashed halfway. Now the view and reality disagree, and you are debugging which one is lying.
The usual fix is more careful update logic. That just makes the copy drift more slowly. The real fix is structural: make the copy disposable, so it can never win an argument with the truth.
Facts, not state
Section titled “Facts, not state”Start from the right source. A command can fail; an event already happened. So the durable record is not “the current state” — it is the append-only sequence of facts that produced the current state.
In our orchestration plane, that record is a literal append-only log: monthly JSONL files under .devportal/events/, written by a single sequencer so there is exactly one writer and one order. Every state change is a fact appended to it — run.recorded, pr.synced, review.summarized, ticket.transitioned, workspace.created. Each fact is validated against a Zod event schema before it is written, and those schemas are forward-only: you add new event kinds and new optional fields, you never rewrite the meaning of a fact already on disk. (That is the append-only discipline from the enum part, applied to history itself.)
Two properties make the log trustworthy as a source:
- Single writer, one order. The sequencer is the only thing that appends, so there is a single, total order of facts — no last-write-wins races.
- Idempotent appends. Each publisher mints an idempotency key, and the sequencer dedups on it, so a retried publish does not become a duplicate fact.
The log is the truth. Nothing else is.
Projections are disposable views
Section titled “Projections are disposable views”Everything you read from is a projection of that log: a Postgres database whose tables (tickets, runs, prs, reviews, runners) are built by replaying the facts. Consumers — active-runs-projection, kanban-projection, parent-lifecycle-mirror — subscribe to the log through a shared consumer framework, read forward from a checkpoint, and fold each fact into their tables.
This is precisely the series’ shape, at the altitude of state-over-time:
the event log (append-only facts) -> the one sourcePostgres projections (tables) -> many generated viewsrebuild-from-log -> the drift gateA projection is never the truth. It is a view of the truth — the same sentence we used for generated code in Part 1: The Drift Problem, now about data. And because it is only a view, it carries no authority of its own.
Rebuild-from-log is the gate
Section titled “Rebuild-from-log is the gate”Here is the part that makes drift recoverable instead of catastrophic.
Because every projection is a pure fold over the log, it can be dropped and rebuilt from the facts at any time. The devportal_state_db_rebuildFromLog operation, run in full or incremental mode, hydrates the projection database from disk — a full rebuild from the start of history, or an incremental catch-up from a checkpoint.
That single capability changes the failure mode completely. When a projection and the log disagree, you do not investigate which is right. The log is right, by definition; you rebuild the projection and the disagreement is gone. A corrupted read model is not a data-loss incident — it is a re-run. The consumer framework keeps durable checkpoints (consumer.offset_committed is itself a fact in the log), and distinguishes those from ephemeral liveness signals like heartbeats, so a restart resumes exactly where it left off without replaying what it already folded.
Drift between truth and its views is not prevented here so much as made cheap to erase. Which, for a copy, is the strongest guarantee there is.
The velocity payoff
Section titled “The velocity payoff”The check this removes is the one that haunts every cache and read model: is this view actually consistent with reality, and if not, how do I fix it without losing data?
When the view is a rebuildable projection, that question stops being scary. You can:
- Add a new read model for free. A new screen needs a new shape? Write a new consumer that folds the existing log into it, and replay history into the new projection. No backfill migration, no “where do I get the historical data” — the history is the log. Standing up a new projection is a replay, not a migration project: the economics invert, and a read model that used to cost a schema change plus a data backfill now costs the time it takes to fold the log.
- Change a projection’s shape without fear. Reshape the table, drop it, rebuild. The facts are untouched.
- Recover from a bad deploy by replaying, not by archaeology.
New views become cheap because truth is one append-only log rather than N mutable tables you must keep mutually consistent by hand. That is the velocity unlock from Part 1: The Drift Problem in its purest form: the structure (an append-only log of validated facts) makes a whole class of expensive coherence work — keeping every read model in sync with reality — simply disappear.
Where the boundary sits
Section titled “Where the boundary sits”This pattern governs our orchestration plane — runs, tickets, reviews, the devportal’s view of the fleet. It is the right tool there because that domain is intrinsically a stream of events about work happening over time.
It is not how the product’s own data works. Workouts, programs, and media are classic CRUD over Postgres, accessed through the repo seam from Part 7: Models at Boundaries — no event sourcing, because that domain does not need replayable history to do its job. Using the heavier pattern everywhere would be its own kind of drift: architecture that has wandered away from what the domain actually requires. We apply projected truth where facts-over-time is the domain, and plain storage where it is not.
What’s next
Section titled “What’s next”We have now seen one source → many projections → drift gates applied to values, models, and state-over-time. The next three parts climb out of the data layer entirely and find the exact same shape governing operations, agents, and the FE↔BE mirror. Part 9: One Operation, Three Interfaces starts with operations: one handler, defined once, exposed as REST, MCP, and CLI at the same time.