Eighty-nine per cent of teams running agents have observability. Fifty-two per cent run evaluations. That gap — between watching a system and knowing whether it works — is the reason a large share of agentic projects are expected to be cancelled rather than shipped.
I run an agentic product — early, in controlled release, and honest about that — under three constraints most teams would call impossible together: personal data never leaves the machine that holds it, the model bill is zero, and no mechanism is allowed to claim it works without two independent proofs. This is what those constraints forced, and what the third one caught.
Being early is why the third constraint exists rather than a reason to discount it. A verification discipline adopted after a system is large is a migration. Adopted before, it is just how the thing is built.
The constraint
The product is a hiring platform. Candidate records are the most sensitive data in the domain, and hiring decisions are the kind a person has to answer for afterwards. Both facts were fixed before any architecture existed.
That ruled out the ordinary shape of an AI product, where records are sent to a hosted model and the answer comes back. Not because it would not work — because a candidate’s history would leave the machine, and because the cost of being wrong lands on a person who never agreed to be processed.
So the constraints were written down first, as things that could not be traded:
- Personal data stays on the device. The product’s reasoning runs on a local model. Voice never leaves the machine either.
- Free, open-weight models only. No paid model path exists in the code at all.
- Nothing claims to work without two proofs. A mechanism that cannot show evidence it fired does not officially exist.
The first two are product decisions. The third is the one this piece is about, because it is the one that turned out to be load-bearing.
What made it hard
An agentic system is not a program that runs. It is a set of mechanisms that are supposed to fire — a cache that should warm, a gate that should refuse, a prefetch that should land before the next turn. Each one is invisible when working and equally invisible when dead.
The ordinary defences do not hold here.
Unit tests pass over dead code. A test can exercise a mechanism’s internals perfectly while nothing in the running system calls it any more. The test stays green. The mechanism is gone.
Monitoring shows traffic, not correctness. You can watch requests arrive, latency behave and errors stay flat while a component quietly stopped contributing anything.
And the failure is silent by construction. Nobody files a bug saying a speculative prefetch stopped being used. The system gets slower in a way that looks like weather.
This is the same shape as a control that reports green over a dead wire — a pathology I first met in a settlement system, where a report disagreeing with the ledger by a single cent was an incident rather than a defect. Twenty years later the mechanism is different and the failure is identical.
What I decided, and what I rejected
Rejected: trust the tests. The obvious answer is a test suite over each mechanism. It fails for the reason above — a test asserting internals cannot see that the call site moved. So the commit-time proof asserts the live call site, not the mechanism’s internals. If the wire is cut, the assertion is what breaks.
Rejected: trust the deployment. A green pipeline says the code built. It says nothing about whether a mechanism fired in the running system. So the second proof is a deploy-time assertion that the thing actually happened — a journey that exercises it end to end, or a counter that must move.
Rejected: a single proof. Either alone is defeatable. A call-site test can be green while the mechanism does nothing useful; a runtime counter can move for the wrong reason. The two are cheap because they are independent, and independence is the whole value.
Rejected: allow-by-default on cost. The zero-cost invariant is a single predicate, and its default matters more than its logic: a provider whose free-ness is not asserted is treated as not free. Silence cannot violate the invariant. The alternative — assume free unless told otherwise — puts an unbounded bill one configuration mistake away.
Rejected: coupling the runtime. The agent runtime and the system of record never import each other. A single tool contract is the only seam between them, which means the agent runtime is swappable — a different orchestrator would work identically. That decision cost real convenience and bought the ability to replace half the system without touching the other half.
Rejected: a green check I could have taken. More on this below, because it is the one I would want to be judged on.
How the register is enforced
Every mechanism in the system is registered when it is built, and the register is not documentation — it is enforced.
Each entry carries two references: the commit-time assertion on its live call site, and the deploy-time evidence that it fired. A guardrail script fails the build when either reference stops existing. That is the part that matters. A register nobody checks decays into fiction within a quarter; this one cannot, because the build breaks when it drifts.
Where a mechanism has no honest proof yet, the entry says so in a marker that names the debt. Those markers are printed loudly on every run. They are not hidden, and they are not optimistically closed.
The register is therefore three things at once: an inventory of what exists, a test that the inventory is true, and a visible list of what is not yet proven.
What it caught
The gate caught itself.
The deploy-time proof originally asserted that a counter was non-zero. That assertion was green over a dead wire, because the counters are lifetime totals seeded from disk at startup: a mechanism that fired once, months earlier, kept the gate green forever afterwards. The check was measuring history and reporting it as health.
It now asserts a delta — the counter must be strictly greater than a baseline captured before the run. A mechanism that stopped firing yesterday fails today.
I want to be precise about what happened there, because it is the argument for the whole design: the verification system was lying, and the thing that exposed it was the verification system’s own discipline. Nothing outside noticed. No user complained. It was caught because the register forces the question what is the evidence this fired? to be asked in a form that can itself be wrong.
And the decision I would most want examined: at one point I could have closed an open row by asserting the easier half of a metric. It would have passed. It would also have been green over a mechanism that was still dead — the exact pathology the delta assertion exists to kill. The row is still open, and the reason is written next to it.
A verification system you are willing to satisfy cheaply is not a verification system.
Costs and limits
It is slower. Every mechanism carries an obligation to prove itself twice, and building one takes longer than building one that merely works. That is the price, and I have paid it knowingly.
It does not prove correctness. Both proofs answer did this fire? — never was the answer right? A mechanism can fire faithfully and produce a bad result all day.
It does not survive a dishonest entry. The register is only as good as the discipline of whoever writes to it. Someone can register a mechanism with a weak assertion and the gate will happily stay green. It reduces silent decay. It does not manufacture integrity.
Some proofs cannot be staged. A latency breaker only fires against a genuinely slow system, which a hermetic test cannot conjure. Those rows stay open rather than being closed with something that passes for the wrong reason.
And the honest limit on the whole design: it tells me a system still does what it did. It cannot tell me that what it did was ever a good idea.
What this means for agents
The industry has spent two years learning to watch agents and has not yet learned to judge them. Observability is near universal; evaluation is roughly half. Quality is the most cited barrier to putting agents into production, and the projected cancellation rate for agentic projects is largely a story about controls that were never built.
The register is not an evaluation framework, and I would not sell it as one. It answers a narrower question — is this mechanism still alive? — which is the question underneath evaluation and the one most systems cannot answer at all.
The transferable part is not the tooling. It is the standard: a mechanism that cannot show it fired does not get to claim it works, and a check you would not let fail is not a check.
That standard does not belong to hiring, or to agents, or to this decade. It belongs to any system where being wrong is expensive and someone has to sign.
The patterns behind this are published as references: agentic approval gates, the agent checkpoint contract, and the handoff pattern.