From ProofAgent Evaluation to Evidence
How a run becomes something governance can use.
Once you know what you want to test, the Harness produces the evidence.
It is an open-source Python package that runs where your agent runs — on a laptop, in CI, inside a VPC. Its job is to drive the agent through designed pressure, screen the things code can decide, judge the things it cannot, and write a structured record that a governance process can consume.
What goes in#
Five inputs shape a run. The agent under test, exposed as something the Harness can call. The agent’s own context — system prompt, tool schemas, and optionally its knowledge. A domain-knowledge corpus, used as ground truth when grading factual claims. A trap library of adversarial scenarios spanning eleven attack families. And an evaluator model, which does the planning, conducting, and judging.
That last input is worth choosing deliberately, because the evaluator is part of the instrument.
What happens during a run#
The Harness plans a campaign against the agent, selects traps weighted toward the risks the governance profile declares, and holds a multi-turn conversation that escalates as it goes.
Alongside the conversation it plants sentinels — markers derived from the run seed rather than randomly, so that a boundary crossing becomes decidable by string comparison rather than by opinion. If a sentinel value appears where it should not, nobody has to adjudicate. The same mechanism catches phantom tool calls, forbidden tool calls, and crashes, which are reported as critical operational defects independent of any judge.
Grading uses several jurors with different dispositions — rigorous, lenient, contrarian — whose verdicts are then reconciled.
There is a practical reason for the extra calls. A single grader is a noisy instrument: the same transcript can score differently on re-evaluation, and a confidently worded answer moves one judge further than it should. Reconciling several jurors damps that variance, and in our experiments adversarial multi-juror scoring surfaced failures that static benchmarks and single-evaluator scoring missed. A jury also leaves a record of disagreement, which is evidence in itself — a finding two of three jurors flagged deserves different treatment from one they agreed on unanimously.
What a jury does not buy is calibration. Reconciling jurors reduces the variance of the instrument; it does not tell you how a score maps to real-world failure rates, and it cannot correct a bias every juror shares.
What comes out#
A run produces a JSON report and a Markdown report. The JSON is self-describing: metric scores, a per-turn trace, findings tied to the turns that evidence them, the context assessment when it was requested, a compliance mapping when it was requested, the readiness index with its decomposition, and an exit code that CI can act on.
Adding a compliance assessment maps the run onto per-control statuses across the frameworks in scope. The catalog holds thirty; the default selection is the EU AI Act, the NIST AI Risk Management Framework, ISO/IEC 42001, and SOC 2. What that produces is a defensible mapping from observed behavior to control statuses, which is genuinely difficult to assemble by hand — and not a certification of legal compliance, which no tool can provide.
Getting a first run working#
Installation and the agent contract are deliberately small.
python -m venv .venv && source .venv/bin/activate
pip install proofagent-harnessThe agent under test is a Python callable named agent. Returning a plain string is enough to run, but returning the tool calls as well is what makes the evidence useful, since an agent can produce calm, correct-sounding prose while calling the wrong tool.
from proofagent_harness import AgentResponse
def agent(message: str) -> AgentResponse:
reply, tool_calls = my_agent.respond(message)
return AgentResponse(text=reply,
tools_called=tool_calls)A first run then looks like this.
proof run agent.py \
--turns 15 --seed 42 \
--assess-context \
--json report.json --markdown report.mdThe aim of a first run is not a good score but a trustworthy pipeline. A short run surfaces credential, timeout, and adapter problems, all of which otherwise masquerade as bad behavior. Run it twice unchanged before interpreting anything: the gap between those two runs is your noise floor, and every later comparison lives inside it.
Two options matter more than the rest. The turn budget drives coverage, since a short run leaves most of the eleven attack families unprobed. The seed makes trap selection and scoring reproducible, so two runs of the same command are comparable.
Where your data goes#
The Harness runs in your environment and does not upload a governance run to the ProofAgent Platform unless you ask it to. There is no telemetry.
The evaluator model is a separate question. If you configure a hosted model as the evaluator, evaluation content including transcripts is processed by that provider under your own agreement with them. Where data must remain inside your environment, run the evaluator locally — any LiteLLM target works, including Ollama, vLLM, and LM Studio.
How evidence reaches governance#
Uploading a run posts it to the Governance API, which resolves the policy in force for that agent and returns a decision. The command then exits on that decision, so a pipeline can gate on governance rather than on a parsed score.
Two details make this workable in practice. Tier the runs: deterministic screens and a small regression pack on every pull request, targeted adversarial scenarios on merge, the full suite with context and compliance assessment for a release candidate, and the full suite again on any material change to the composition record, even when there is no code diff. And hold the evaluation protocol still while comparing versions, since a change to the evaluator model or the trap packs moves scores even when the agent has not changed.
The appendix carries the fuller command reference. What matters here is the shape: evidence is generated where the agent lives, and travels to governance as a structured record rather than as a screenshot of a number.
Evidence generated where the agent runs, and recorded in one structure, is what makes a governance decision possible.
Get one evaluation running end to end before worrying about the result. A first run is a test of the pipeline, and the numbers only become meaningful once you know how much they move on their own.