Building an Evidence-Based Control System

Controls that can fail, tests that reveal it, and artifacts that prove it.

A control is worth writing down when three things are true of it. There is a state of the world in which it is not satisfied. Somebody can run something and find out. And the running leaves an artifact a reviewer can inspect afterward.

Statements that miss any of the three are usually values rather than controls, and values belong in the policy preamble. “The system prompt instructs the agent to follow policy” is satisfied by the existence of the sentence, so it produces a green cell forever and tests nothing. “The team exercises appropriate judgment” may well be true, and cannot support a release decision.

Three kinds of control, acting at three different moments#

Controls reduce different things, which is why a high-risk agent needs more than one kind.

A preventive control reduces the chance that something happens: a least-privilege tool scope, an approval requirement enforced in the tool layer. A detective control reduces the time to find out: an injection detector with an alert attached. A responsive control reduces the damage afterward: a revocation path that somebody has actually tested and timed.

None substitutes for the others. Prevention that fails silently is indistinguishable from no control at all, and detection without a response path produces alerts nobody can act on.

The families that matter for agents#

Seven areas cover most of what an agent program needs.

Human approval is a named role acting before a specified action proceeds — not “a human reviews,” but two named roles recording an approval that the action cannot execute without. Access control is the identity the agent runs as, scoped to its job, and it is the only control that holds regardless of what enters the context. Action authorization is tool-layer enforcement: thresholds, allowed operations, required arguments, gates on irreversible steps.

Data controls classify fields and decide which classes may reach which tools. Tool controls register scope and declared side effects, and enforce a deny list rather than describing one. Context controls separate trust at assembly, require provenance on high-impact facts, and demand grounding before a claim of a given class is produced. Re-evaluation controls take the change triggers from the composition record and wire them to something that actually forces a new run.

Four of those seven are infrastructure decisions from Part II. A control, in that sense, is where an infrastructure decision becomes a governed commitment.

One control, written properly#

The most useful format is a repeated card, because the repetition is what makes a catalog reviewable.

CONTROL FIN-07

Purpose — Prevent fraudulent redirection of vendor payments.

Control — Vendor bank-account changes require verified source data and dual human approval, enforced in the tool layer.

Type — Preventive, with a detective companion (FIN-11: injection attempt logged and alerted).

Test — Adversarial scenario including an injected waiver attempt, plus a direct tool-layer check with the approval records absent.

Evidence — Turn-level trace, tool-call record, approvals table rows.

Owner — AP systems lead.

Threshold — Zero successful waivers. No partial credit.

Status — Failed at v2.3.0; passed at v2.3.1.

Failure effect — BLOCK the payment agent’s release.

Two conventions make a catalog easier to live with. Name controls after behavior rather than after a framework clause, since “external write actions require approval” is more actionable than “human oversight control 3.2.” And keep the framework reference as metadata, because one well-designed control often satisfies several obligations, and duplicating it per framework guarantees the statuses will disagree.

When the right control is not available yet#

Sometimes the control you want cannot be implemented in time. The workable response is a compensating control with an expiry: a narrower scope, a lower threshold, a human in a path that would otherwise be automated, or a monitoring rule that catches what prevention cannot.

What separates that from a quiet bypass is the record — who approved it, what it compensates for, which scope it covers, when it expires, and what has to be true to close it.

Controls tell you what must hold. Governance determines what happens when they do not.

A control you cannot test is a statement of intent, not a safeguard.

Apply This Chapter

Audit your existing controls against the three-property test, then fill the most obvious gap. Teams usually find that their catalog is heavily preventive and almost entirely undetectable.

Get these as working templates