The gate was green, but the mechanism it was supposed to verify had stopped firing.

The check was reading a lifetime counter. A mechanism that had run months earlier could stop today and still pass. We were checking history and calling it health.

That failure sharpened a standard I use in my own engineering: before I count a mechanism as working, I want evidence that the live system is connected to it and evidence that it actually ran.

The constraints came first

The product is an agentic hiring platform, early and in controlled release. Candidate records are sensitive, and hiring decisions affect people who may never see the systems behind them. Those facts shaped the design before implementation.

I set three constraints:

  • Personal data stays on the device that holds it. The product’s reasoning runs on a local model, and voice stays on the machine too.
  • Free, open-weight models only. No paid model path exists in the code.
  • Every mechanism needs two independent proofs. It must show that it is connected to the live system and that it fired in a running system.

Being early made this the right time to establish the discipline. Adding it later would mean revisiting mechanisms already treated as complete.

The cost constraint also needed an explicit default. A provider is treated as not free unless its free status is asserted. Assuming the opposite would let an incomplete configuration break the constraint.

I also kept the agent runtime separate from the system of record, connected through a single tool contract. Neither imports the other. That costs some convenience, but it keeps the runtime replaceable without requiring a platform rewrite.

Why a green test is not enough

A cache should warm. A guardrail should refuse. A prefetch should arrive before it is needed. When these mechanisms work, they can be almost invisible. When they stop, the system may continue without an obvious error.

A unit test can exercise a mechanism’s internals perfectly after the application has stopped calling it. Monitoring can show healthy traffic while a component contributes nothing. A successful deployment proves that code shipped, not that a particular behaviour occurred.

I had seen the same shape of problem in settlement systems: a healthy-looking report could disagree with the record underneath it. That experience made me wary of accepting a control’s status as proof of the thing it watched.

What each proof has to show

The first proof checks the live call site. It belongs close to the code and should fail if the mechanism is disconnected. Testing the internals alone does not answer that question.

The second proof checks execution in the running system. It might be an end-to-end journey or a counter that moves during the current run. It must show that the mechanism actually fired after deployment.

Neither is sufficient alone. A call site can exist without useful execution. A runtime signal can move for the wrong reason. Their value comes from checking different things and being able to fail independently.

Each registered mechanism carries both references. A build check fails if either reference disappears. Where honest proof is still missing, the register makes that unfinished work visible and names what is owed.

The register gives me an inventory, a check on its evidence references and a record of what I cannot yet claim. It does not relieve me of judging whether the evidence is meaningful.

The check had to change

The lifetime counter exposed the difference between having a measurement and asking the right question.

A non-zero total showed that something had happened at some point. It could not show that the mechanism fired during the journey we were checking. Restoring that total from disk preserved the old success even after the current behaviour stopped.

The proof now compares the counter with a baseline captured before the run. The value has to increase during that run. A mechanism that fired last month but not today fails today.

What mattered was the discipline of asking a precise question: what is the evidence that this mechanism fired in this run? That question made the weakness in our own check visible.

The check I chose not to accept

There was another decision I would want a reviewer to examine.

An open entry could have been closed by asserting the easier half of a metric. The assertion would have passed while the part that mattered remained unproven. I left the entry open, with the reason recorded.

The temptation is understandable. A green check clears the queue and makes progress easier to report. But if I accept evidence because it is convenient, the automation will enforce that weaker standard faithfully.

Keeping unfinished work visible is part of the design. It is also a choice I have to keep making.

What the proofs do not establish

They do not prove correctness. A mechanism can be connected, run reliably and still produce a bad result. These proofs establish connection and execution.

They depend on the quality of the entry. A weak assertion can satisfy the register. The system reduces silent decay; it cannot make a meaningless test useful.

Some behaviours are hard to stage honestly. A latency breaker needs a genuinely slow system. If a controlled test does not reproduce the condition, I cannot treat its success as that proof. The entry stays open.

There is a cost too. Each mechanism takes longer to build because it carries an obligation to prove itself twice. I consider that worthwhile, but it is engineering work that has to be planned.

Observing and evaluating are different

The industry figures behind the original version of this article help explain why I care about the distinction. LangChain’s State of Agent Engineering survey, fielded in November and December 2025 with 1,340 respondents, reported roughly 89% using observability and 52% running evaluations. Gartner’s June 2025 forecast also anticipated substantial cancellations of agentic projects. These are external survey findings and a forecast, not measurements of my product.

They provide context, not proof of this design. My register is not an evaluation framework either. It answers a narrower question beneath evaluation: is the mechanism still connected, and did it run?

That is the claim I am prepared to make. Whether its output is correct, and whether building it was a good decision, still require separate judgment.

A mechanism without honest proof stays unfinished. I would rather defend that open entry than explain why I accepted a green check I knew was insufficient.

The patterns behind this are published as references: agentic approval gates, the agent checkpoint contract, and the handoff pattern.