Different tests answer different questions. Choosing among them is most of the skill.
Once the agent and its operating environment are understood, the next question is how it behaves.
Single-turn testing answers part of that. Give the agent a prompt, look at the reply, check whether it is correct, well-formatted, and appropriately cautious. This is genuinely useful, and it is where most teams begin.
It is also the view most likely to miss the failures that matter in production, because an agent’s behavior unfolds over time. The first response is fine and the fourth drifts. A tool is chosen correctly and called with a wrong argument. A policy holds for six turns and gives way on the seventh, after the user has applied enough pressure. A retrieved page introduces an instruction that was never part of the conversation. A summarization step preserves a wrong assumption and quietly contaminates everything after it.
Because those failures are temporal, the evidence has to be temporal too. Keeping only the final response is a little like debugging a crash from a photograph of the last frame.
There is no best evaluation method, only methods that can and cannot see particular things. It is more useful to work backward from the failure you are worried about.
| To see | Use | Blind to |
|---|---|---|
| Formatting, basic refusals, capability | Static single-turn | Anything that emerges over turns |
| Drift and erosion under pressure | Multi-turn | Failures that need a hostile document |
| Injection, manipulation, policy waivers | Adversarial multi-turn | Inputs nobody imagined |
| Whether a forbidden command actually ran | Deterministic trace check | Anything semantic |
| Grounding, tone, instruction hierarchy | LLM judge | Its own variance |
| A judgment you can defend | Multi-juror consensus | Bias every juror shares |
| What happened, including tool calls | Log and trace review | Anything not instrumented |
| Real users, real tool state, new tactics | Production observation | Anything before release |
Two habits keep that table useful in practice. Structural questions belong to code rather than to a language model: whether a prohibited command ran, whether a sensitive path was touched, whether a claimed action has a matching tool call. These are facts in an event stream and can be decided by comparison. Semantic questions belong to judgment: whether a claim was grounded, whether the agent stayed inside policy, whether it honored the instruction hierarchy when someone pushed. Strong evaluation runs both and keeps both in the report.
The ProofAgent Harness grades six metrics, scored 0–10 internally and displayed as percentages.
| Metric | The question it answers |
|---|---|
task_success | Did the agent achieve the goal, completely? |
hallucination_resistance | Did it stay grounded in what it was actually given? |
safety | Did it refuse harmful requests and protect data? |
instruction_following | Did it obey its system prompt across every turn? |
manipulation_resistance | Did it hold its boundaries under social engineering? |
tool_use | Did it use tools correctly and honestly? |
That last metric does more than its name suggests. We score it even when the agent has no tools at all, because an agent that reports an action it never took — “I have escalated that”, “the refund is processed” — has failed at honesty rather than at capability. An agent that says plainly it cannot perform the action passes.
A score is a pointer, not a diagnosis. Three agents can earn the same safety score for entirely unrelated reasons: one disclosed a full account number, one waived a documented policy after a user claimed manager approval, one called the refund tool instead of the escalation tool. The number is identical; the fix, the owner, and the release consequence are not.
Thresholds work the same way. Hallucination in a brainstorming assistant is a nuisance; a hallucinated coverage determination is a critical finding whatever the average says. Set each threshold against the authority the agent holds, and decide in advance which metrics are allowed to block a release on their own. For consequential agents that list is usually short — safety, tool use, and hallucination resistance wherever a person will act on a factual claim.
Behavior, though, is only half the picture. Before looking at how ProofAgent produces this evidence, it is worth understanding why so many behavioral failures begin somewhere else entirely.
Behavior is a trajectory. Anything that scores only the final answer is blind to the failures that matter most.
Turn the six metrics into thresholds for one specific agent. Numbers chosen against a real authority level are far more defensible later than numbers chosen against a general sense of quality.