The most expensive sentence in an agent system is “it shipped.” A change can exist without being exercised, work without being visible, or look current while describing a different system. The repair is to separate the claim from the proof.
By Zion Boggan ·
A status is a claim. An artifact is evidence. A working path is proof.
One security change passed its build because the compiler did not include the directory that changed. A client-facing fix landed in source control while the real user path still failed. An upload service accepted a file while the watch page remained unavailable. A status board said finished files existed, but the files lived on another machine.
These are useful beginner examples because each starts with a true sentence. “The command passed.” “The commit exists.” “The upload returned success.” The mistake is letting a narrow truth stand in for the larger outcome.
Each report was technically true in a narrow sense. None answered the question a person actually meant: can the user use the thing now?
Ask for four things in order: the precise claim, the exact artifact, the smallest real behavior check, and the result from the user surface. “The fix shipped” is too broad. “The empty-email validation is present in the form component and its test” is checkable.
| Question | Evidence |
|---|---|
| What is being claimed? | One sentence with a bounded subject and outcome. |
| What exists? | Path, version, hash, receipt, commit, or record identifier. |
| What runs? | The smallest test that exercises the real behavior. |
| What can a user observe? | Page, response, device state, delivered artifact, or other relevant surface. |
The missing box is a specific unverified state, not a reason to guess. A useful handoff says exactly which state is proven and which is not.
The obvious answer to stale data is “use the newest file.” That is not enough. Two systems can both be current and still describe different realities. In one audit, an analytics collector and a posting router each held recent records, but they shared no item identifiers. The dashboard was not old. It was measuring a different ledger.
The right question is not “which timestamp is newer?” It is “which component controls the action I am asking about?” For a live identity, ask the live identity service. For a scheduled job, inspect the scheduler. For a published page, load the page. For a render, find the file where the renderer writes it. A handoff can explain history, but it cannot overrule the system that currently acts.
Moving a rule out of prose and into machinery is usually an improvement. It makes the boundary real. But machinery has no sense of time. A stale instruction enforced by a hook can block the action that would reveal the instruction is wrong.
In one incident, a memory note said a compute host had been deliberately stopped. A pre-command guard enforced that sentence and blocked a start attempt. The machine told a different story: the host had been running for seventeen days, routing work, and keeping a resident heartbeat process alive. The stale memory had become a mechanical veto against observable reality.
The same guard also blocked a warning because the warning contained words associated with a prohibited power action. Natural-language substring matching is not a policy language. It cannot reliably distinguish an instruction from a description of an instruction.
A safer version has a narrow rule such as “block this exact command when the target matches this approved list.” It does not search every paragraph for a dangerous word and then guess what the writer intended.
The lesson is not to avoid hooks. It is to give them the same evidence discipline as agents: known scope, test cases, review date, and a failure mode that does not turn yesterday’s truth into today’s blind spot.
A transcript file is a poor heartbeat. It can stay quiet while a browser agent clicks, a remote process renders, or a worker waits for an external response. It can change after the work is finished. A watcher built on transcript silence produces false alarms and missed stalls.
Choose a signal that belongs to the work: a command timestamp, child process, artifact write, service health response, or state transition with a receipt. Classify the result before alerting. Active, completed, waiting, and failed are different states. “No new chat text” is none of them.
Once a pipeline has consumed days of work, every new measurement feels like a threat to the work already spent. A plausible explanation is easy to defend. A bounded comparison is harder to argue with.
State the claim, choose the smallest test that could disprove it, and let the result decide whether the current design survives. A blind comparison in one production pipeline selected the replacement framing in every sample, and its measured render cost was lower. That result retired the old design. The scorer fixes were not wasted, but they stopped being a reason to keep the wrong shape alive.
If a test cannot change your mind, it is not a test.
Show me the artifact, exercise the real path, observe the result from the user’s surface, and name anything that remains unverified.
That costs more than a green checkmark. It also prevents the expensive version of success: telling someone that a fix works, making them discover that it does not, and leaving the next operator a stale record that repeats the claim.