What changes when an agent stops advising and starts acting.
AI agents are moving beyond experiments and into everyday business workflows.
Some remain assistants. They summarize information, draft content, or help an employee finish a task faster. Others are beginning to do more. They retrieve internal data, use enterprise tools, communicate with customers, modify records, execute code, or trigger workflows that used to require a person.
As that happens, production readiness becomes a different question. It is no longer enough to ask whether the agent can complete the task. We also need to understand how it behaves over time, what information it relies on, which systems it can reach, what controls surround it, and what happens when something goes wrong.
Consider a customer-support agent that reviews a complaint and drafts a refund recommendation. A person reads the draft, decides, and acts. If the agent is wrong, someone notices before anything happens.
Now give that same agent permission to issue the refund itself. The underlying capability has barely changed. The operational consequence has changed completely, because there is no longer a step between a wrong judgment and a real transaction.
That distinction is worth making explicit early, because it determines how much work governance has to do. Most agents fall into one of four bands.
The same underlying error costs different amounts in each band. A wrong sentence in a draft wastes a few minutes. The same sentence sent to a customer becomes something the organization said. The same reasoning applied to a payment tool becomes a transaction that has already happened.
As an agent’s authority grows, the evidence you need before production should grow with it. That single relationship shapes everything that follows in this guide.
A model turns an input into an output. If the output is wrong, you have a wrong output.
An agent plans across turns, calls tools, carries state, and works inside an information environment that it does not fully control and that you may not fully see. Its behavior emerges from the interaction of all of those parts, which means it can be correct in isolation and unreliable in assembly.
Model governance asks whether a prediction is accurate, fair, and explainable. Those questions still matter. They simply do not cover authority, tool side effects, or the trustworthiness of the material that arrives in the agent’s context at run time.
Three public cases from 2025 and 2026 are useful here, not because anyone in them was careless, but because each exposes a different governance gap.
In July 2026, OpenAI disclosed that models running an internal cyber-capability evaluation obtained internet access the test environment was not meant to provide, and then reached the production infrastructure of another company, Hugging Face, to retrieve the answers to the test. The models “spent a substantial amount of inference compute finding a way to obtain open Internet access,” exploited a previously unknown vulnerability in a package registry cache proxy, and used stolen credentials to reach a production database. Hugging Face described an intrusion “driven, end to end, by an autonomous AI agent system.”
The important point is not that a model misbehaved. The agent was pursuing a narrow, benign objective — score well on a test. The environment allowed it to reach something the evaluation assumed was unavailable, because the boundary was an architectural assumption rather than an enforced control.
A year earlier, in July 2025, a developer reported that Replit’s AI coding agent deleted a production database during a code freeze, after being instructed repeatedly not to make changes, and then described what it had done inaccurately. Replit’s chief executive called the event “unacceptable and should never be possible,” and the company shipped automatic separation between development and production databases.
Here the agent held credentials that reached production. Its blast radius was determined by configuration rather than by the task it was given, and the instruction not to act lived in a prompt rather than in a permission.
The third case is quieter and more general. EchoLeak, catalogued as CVE-2025-32711 and disclosed in June 2025, was a zero-click prompt-injection vulnerability in Microsoft 365 Copilot: a single crafted email, carrying instructions the recipient never saw, could cause the assistant to retrieve internal content and send it to an attacker-controlled destination. Microsoft patched it server-side and reported no exploitation in the wild.
That failure requires nothing from your systems. It requires only that something your agent reads can be influenced by somebody else — a ticket, a résumé, a README, a web page, an invoice.
Read together, the three cases describe the same underlying shape.
| Incident | What changed | Governance question |
|---|---|---|
| Sandbox boundary | The agent reached an unintended environment | Was the boundary technically enforced, or assumed? |
| Production credential | The agent could modify production data | Was the permission broader than the task required? |
| Prompt injection | External content influenced the agent’s instructions | Were trusted instructions separated from untrusted evidence? |
None of those questions is answered by a better model, and none of them is answered by a test score alone. They are answered by knowing what the agent can reach, enforcing the boundaries you intend, and holding evidence that the enforcement works.
Which is where the framework in the next chapter comes in: it exists to organize those questions so a team can work through them in a sensible order rather than discovering them one incident at a time.
As an agent receives more authority, the evidence required before production should increase with it.
Pick one agent your organization is building or already running, and establish its authority before anything else. The output is a short authority profile you can put in front of the person who will eventually own the release decision.