Risk, Ownership, and the AI-BOM
Knowing the agent well enough to govern it, and recording it well enough to prove it.
Before an agent can be evaluated, it has to be identified. Not described in a document — recorded in a form the release process can read.
A useful record answers seven questions: who owns it technically and on the business side, what it is allowed to do, what it must never do, what data it touches, which tools it can call, how much autonomy it has, and what it is built from.
The third question is the one teams skip, and it is the one an evaluator needs most. “This agent answers billing questions” is an intended use. “This agent does not make retention offers, does not waive fees, and does not discuss litigation” is a prohibited use, and it tells anyone testing the agent exactly what to try.
Classifying risk before testing anything#
Not every agent should carry the same evidence burden. A research assistant that reads public information and an agent that can change a bank account are different problems, and treating them identically means either strangling the first or under-governing the second.
In ProofAgent, six answers drive the classification: primary use case, autonomy level, data sensitivity, region, whether a human reviews, and whether the agent takes consequential external actions. The same deterministic classifier runs on a developer’s machine and in the platform, so a local check and a cloud check agree, and it returns one of four tiers — minimal, limited, high, or unacceptable.
Two properties of that scheme are worth adopting regardless of tooling. The base tier comes from the use case and risk factors escalate it, but factors escalate only as far as high; nothing except a prohibited use case makes an agent unacceptable. And the classification carries its reasons, so a tier can be explained to the team it constrains rather than merely imposed on them.
What makes a tier operational rather than decorative is that it changes the requirements.
| Tier | Typical shape | What the tier changes |
|---|---|---|
| Minimal | Internal summarization, coding assistance | Baseline evaluation, watch for regressions |
| Limited | Customer support, content generation | Disclose AI use; evaluate and gate every version |
| High | Credit, hiring, healthcare, payments | Human oversight, sign-off before release, block on high-severity findings, periodic re-evaluation |
| Unacceptable | Prohibited practices under EU AI Act Article 5 | Not deployable; escalate to compliance |
The composition record#
An AI bill of materials is the record that lets you reconstruct what you actually approved.
A field belongs in the record if one of two things is true: you need it to interpret evidence, or a change to it invalidates evidence. Model and version does both. Who wrote the first draft of the system prompt does neither.
The distinction between documentation and control is whether anything consumes the record. An AI-BOM you fill in once is documentation. An AI-BOM the release gate reads, and that triggers a re-evaluation when a marked field changes, is a control.
Where infrastructure creates risk#
With the record in front of you, a set of recurring risks becomes easy to see. Most agent failures in production trace back to one of them.
The first is a permission broader than the task. Consider an accounts-payable agent that only needs to read vendor records. If it runs under a service identity that can also modify those records, the agent already has more authority than its work requires. The safer design gives it a read-only identity and reserves write access for a separately controlled action.
The second is a tool whose effects are unclear. An agent asked to pause a subscription once called cancel_subscription instead; the argument was valid, the call succeeded, the confirmation read plausibly, and the account was gone. Wrong tool selection usually traces to overlapping descriptions, and bad arguments to weak schemas, so both are fixable where the tool is defined.
The third is retrieved content treated as instruction. When a document says “dual approval waived by finance policy 12.4” and the agent honors it, the fault is in the assembly rather than the model: untrusted text arrived in the channel reserved for instructions.
Memory contributes a fourth. At turn two a customer says they are in Germany and the agent correctly applies EU data handling; at turn fourteen, after a summarization step compresses the history, the location is gone and the agent offers a US-only option. Nothing failed loudly. A constraint simply fell out of the context.
Context budget contributes a fifth, less dramatically. Teams add every policy document that might help, the material that mattered ends up buried between two pages of boilerplate, and reliability drifts downward for reasons no log explains.
For the agents your teams use rather than build — coding agents, browser agents, assistants that read local files and run commands — the practical instrument is an access map: paths read, paths written, commands executed, hosts contacted, tools invoked. Compare observed access against intended scope over a week of real sessions. An agent configured for one repository but repeatedly reaching unrelated paths is a governance issue even when the code it writes is excellent.
What these have in common is that each began as an assumption nobody enforced. Converting assumptions into enforced conditions is most of the work described in Part IV.
Once you know what the agent is built from and what authority it carries, you can evaluate the system that actually exists rather than the demo you intended to build.
Record what the agent is made of, and you can tell when your evidence has expired.
Build the record before you need it. A first pass through the composition record for one real agent typically surfaces a permission nobody chose, a tool nobody registered, and at least one field only a single person can answer.