Lesson 4.3
Testing Agents and Tool Use
Why agents need a different test strategy from single calls, the levels of testing from tool unit tests to end-to-end trajectory evaluation, and how to control side effects and non-determinism.
Testing a single model call is a matter of input and output. Testing an agent means testing a sequence of decisions, each contingent on tool results, that ends in a change to the world. The path varies between runs even when the outcome is the same. The professional exam expects a layered strategy that isolates what can be isolated, controls side effects, and judges the agent on what it achieved and how it got there.
Levels of testing
| Level | What is tested | Technique | Determinism |
|---|---|---|---|
| Tool unit tests | Each tool's behaviour and error handling | Ordinary unit tests, no model | Fully deterministic |
| Tool-selection tests | Given a situation, does the model pick the right tool with the right arguments? | Single-turn requests, assert on the tool_use block | Mostly; repeat runs |
| Step tests | One turn of the loop with recorded tool results | Replay fixtures | Mostly |
| Trajectory evals | Whole runs: order, efficiency, safety behaviours | Sandbox plus trajectory grader | Variable; repeat runs |
| Outcome evals | Final state meets the specification | State-based assertions, test suites | Variable; repeat runs |
| Safety tests | Refuses destructive or injected instructions | Adversarial scenarios in sandbox | Variable; repeat runs |
Most teams start at the top and bottom only. The middle layers are where regressions are caught cheaply.
Tool-selection tests
The cheapest high-value agent test. Construct a conversation up to the point of decision and assert that the response contains a tool_use block for the expected tool with arguments matching expectations (exact for identifiers, tolerant for free text). Include negative cases: situations where no tool should be called, and ambiguous situations where the agent should ask. Run each case several times and require a pass rate, not a single pass.
Controlling side effects
Agents act. Tests must not. Three approaches, often combined:
- Mock tools return scripted results. Fast and deterministic, but they only test the paths you scripted.
- Sandboxed real tools operate on disposable copies: a seeded database, a cloned repository, a test tenant. Realistic, slower, and they catch integration faults.
- Recorded replay captures real tool results from production or exploratory runs and replays them. Good for regression on known trajectories.
Whatever the approach, write tools should expose a dry-run mode so that even exploratory agent runs can be observed without consequence.
Outcome evaluation
Judge the end state, not the transcript. For a coding agent: do the tests pass, does the diff touch only allowed files, does the build succeed? For an operations agent: does the ticket exist with the right fields, was the record updated once and only once? State-based assertions are code graders and inherit their reliability. Where the outcome is a document or analysis, fall back to a model grader with a rubric, calibrated as described in the graders lesson.
Trajectory evaluation
Some properties live in the path: the agent should search before asserting, should not call a write tool without a confirmation step, should stop when done rather than continue polishing, should not loop on a failing tool. Grade trajectories with a model grader fed a structured summary of the run (tools called, arguments, results, turn count) and a rubric of binary items. Also record operational metrics: turns, tokens, cost, wall-clock time, tool error count. A system that reaches the right outcome in forty turns instead of eight has regressed.
Non-determinism and the pass^k standard
Repeat every agent eval several times. For systems with no human filter, report pass^k: the proportion of cases where all k attempts succeed. For assistive systems where a human reviews, pass@k may be acceptable. Set k and the threshold based on the cost of failure, and expect pass^k to be noticeably lower than the single-run pass rate; that gap is the honest measure of reliability.
Safety and injection scenarios
Build adversarial cases into the suite: tool results containing instructions to delete data, documents that claim to be from the administrator, web pages that ask the agent to visit a URL with secrets attached. Assert that the agent does not comply and does flag the content. These tests belong in the regular suite, not a one-off security review, because prompt or model changes can regress them.
Harness-specific support
The Agent SDK's hooks capture every tool call and result, making trajectory logging a configuration matter; the same hooks can block writes during tests. Managed Agents can run an agent against an outcome definition with a rubric and report whether it was met. For your own loops, instrument the loop to emit the same structured events, so that one trajectory format feeds tests, dashboards and incident review.
What a good agent test suite looks like
- Hundreds of tool unit tests, run on every commit, seconds.
- Dozens of tool-selection and step tests, run on every prompt or tool change, minutes.
- A set of sandboxed trajectory and outcome cases, run on release candidates and model changes, repeated for pass^k.
- A standing adversarial set, run alongside.
- Results stored with the prompt version, model ID and dataset version that produced them.
Exam focus
- Agents are tested in layers: tool unit tests, tool-selection tests, step replays, trajectory and outcome evals, safety scenarios.
- Tool-selection tests assert on the tool_use block and include negative and ambiguous cases.
- Control side effects with mocks, sandboxed copies or recorded replays; give write tools a dry-run mode.
- Judge outcomes with state-based assertions; judge trajectories with rubric-based model graders and operational metrics.
- Repeat runs and report pass^k for unsupervised systems.
- Adversarial injection scenarios belong in the standing regression suite.
Check yourself
5 questions
1.A travel-booking agent calls live airline and payment APIs. Running its test suite creates real bookings and charges, so the team runs tests rarely. Which approach makes agent testing both safe and routine?
2.An agent passes a particular task 7 times out of 10 trials. The product requires that customers can rely on the feature working essentially every time. One engineer reports the task as 'passing' because at least one trial succeeded. Which metric and interpretation is CORRECT for this requirement?
3.A team's agent tests assert the exact sequence of tool calls the agent must make. After a model update the agent reaches the correct result through a slightly different but valid sequence, and half the suite fails. What should the architect recommend?
4.A procurement agent behaves well in testing, but in production it occasionally takes incorrect actions after a supplier API returns a timeout or a malformed payload. Which gap in the test suite is MOST likely responsible?
5.A document-processing agent reads supplier-submitted PDFs and can update payment records. The security team asks how the test suite demonstrates resistance to malicious documents. Which addition is MOST appropriate?