From Requirements to Testable Controls
Turning an obligation written in prose into a condition a system can check.
A policy may say that high-risk actions require human oversight. Before that statement can govern an agent, the team has to define which actions are high risk, who must approve them, and how the system proves that the approval occurred.
That gap — between an obligation as written and a condition a system can check — is where most compliance effort either becomes useful or quietly stops.
The path across it has five steps: requirement, control, test, evidence, status, and then the decision that follows. Each step removes ambiguity from the one before, and the whole chain can be walked in reverse when somebody asks why an agent was allowed to ship.
What changes when a requirement becomes a control#
“Protect sensitive information” is a requirement. It is also untestable: there is no state of the world in which a system could report that it failed.
“Restricted fields must never be sent to an unapproved external tool” is a control. It can be tested, traced, and blocked on, because it describes a condition that could be violated and an observation that would reveal the violation.
The same translation applies across most of a framework mapping.
| As written | As a control |
|---|---|
| Use AI responsibly | The agent cannot execute state-changing actions outside its registered tool scope |
| Protect personal data | Restricted fields cannot be sent to unapproved tools or external endpoints |
| Keep a human in the loop | Payments above the policy threshold require two recorded approvals before execution |
| Prevent hallucinations | High-impact factual claims require grounding to an approved source, or escalation |
| Monitor the agent | Material model, prompt, tool, permission or retrieval changes trigger re-evaluation |
| Follow policy | A failed critical control produces BLOCK regardless of aggregate readiness |
Notice what happened to human oversight in that table. It was not a control until somebody specified which actions it applies to, how many approvers are required, and whether the approval can be bypassed. Oversight that cannot be observed cannot be evidenced, and oversight that cannot be evidenced tends to erode.
Using frameworks for what they are good at#
Frameworks are best treated as checklists of obligations you might otherwise forget, rather than as evidence in themselves. A handful matter for most agent programs.
The NIST AI Risk Management Framework organizes work into Govern, Map, Measure and Manage, and its practical contribution is insisting that measurement and management are separate activities with separate owners. ISO/IEC 42001 contributes lifecycle discipline: defined roles, documented decisions, internal audit. The EU AI Act contributes consequence, because its risk tiers change what is actually required — prohibited practices under Article 5, and high-risk obligations under Annex III covering risk management, human oversight, logging, and post-market monitoring. OWASP’s agentic guidance contributes a concrete adversary model: injection, tool misuse, identity abuse, memory poisoning, excessive agency. And internal policy is usually the strictest and most specific instrument an organization has, while being the one most often left out of the mapping.
The ProofAgent compliance catalog spans thirty frameworks, defaulting to the EU AI Act, NIST AI RMF, ISO/IEC 42001, and SOC 2, with regional privacy regimes, sector regimes, and the agentic security lists available when they apply.
Being honest about coverage#
A mapping that reports “eleven NIST SP 800-53 controls covered” invites the reader to hear something much larger, since Rev. 5 holds roughly a thousand controls across twenty families.
Two habits keep coverage claims defensible. Where a standard has a definite published size, quote coverage as a fraction of it. Where a denominator would mislead, publish no number and name what was not assessed instead.
Some controls genuinely cannot be observed through behavioral evaluation. A supply-chain control needs registry and dependency evidence. An inter-agent communication control needs a second agent on the wire. Saying so is stronger than an inflated percentage, and it tells the reader where to look next.
It also matters that ProofAgent, or any tool, produces a mapping from observed behavior to control statuses rather than a compliance certificate. Compliance is a determination an organization makes with counsel; evidence is what it brings to that determination.
Five statuses, because two are not enough#
A control can be passing, failing, unevaluated, not applicable, or awaiting a human review. Collapsing that to pass and fail creates a specific and common problem: “we did not test that” and “that is fine” produce identical green cells.
The compliance dimension of readiness enforces the same distinction. It is scored across the controls actually evaluated, and where fewer than six controls were assessed it is withheld rather than reported from a handful of observations. Absence of evidence is not evidence of readiness, and a scoring system that cannot express the difference will eventually be used to hide it.
Testing whether a control holds#
A policy may define dual approval while an API path bypasses it. A prompt may forbid external transmission while the tool layer permits the call. The only way to know is to test the control in the running system.
That is what makes the injected-waiver scenario in Chapter 16 worth building. It does not check whether the control was documented. It checks whether the control held when something tried to waive it — and in the first version, it did not.
Controls tell you what must hold. The next chapter is about designing them so that the answer is knowable.
A requirement becomes useful when you can test whether it holds.
Translate one real obligation into something a system can check. A single requirement carried all the way to a status is usually enough to show a team what the rest of their mapping is missing.