13.4 Capstone 4 — an agent with tools and evals
An agent that completes a real multi-step task, with cost and failure limits.
The final capstone is an agent completing a real multi-step task with tools, cost ceilings and a documented failure taxonomy — proof of engineering autonomy rather than prompting it. It closes the project sequence because it composes the most modules at once. The selection criterion is measurability: a task without a measurable success criterion cannot be evaluated, and it-usually-works is a demo review, not an evaluation.
Work through these
Task selection with a measurable success criterion
A task with a success criterion that can be checked automatically, because an agent you cannot score is an agent you cannot improve. Choosing this well is the hardest part of the capstone.
Tool design and error handling
Tools designed so the model uses them correctly, with sensible behaviour when they fail. Tool errors are the normal case rather than the exception.
Tracing, cost per task, step ceilings
Recording every run, measuring what each task costs, and capping the number of steps. These three are what make an agent safe to leave running.
Trajectory evaluation and a documented failure taxonomy
Evaluating the path as well as the outcome, and writing down the ways it fails. A documented failure taxonomy is more impressive to an interviewer than a success rate.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.