Module 2 · Running benchmarks · Lesson 4
Task specs, harnesses and sandboxes
What a reusable agent task contains (instruction, environment, verifier and reference solution), how Harbor, Inspect and τ-bench each package one, and the oracle and empty-agent controls that prove a task is solvable and not passed for free.
By the end of this lesson you can
- Name the parts of a reusable agent task and say which runtime role (agent, sandbox, verifier, oracle) may see each one
- Package the same stateful task for Harbor (instruction, environment, test script that writes a reward, reference solution) and for Inspect (dataset, solver, scorer, sandbox)
- Run oracle and empty-agent controls and interpret each of the four possible outcomes
- Run Claude Code on a subscription inside a container with a long-lived OAuth token, with repeated independent attempts
- Choose between Harbor, Inspect, τ-bench and a custom harness for a given evaluation