Bonus dive
Exercises: make the learning stick
Reading code teaches you less than predicting what it will do and then checking. This file turns each section of the README into a few quick active-recall prompts.
How to use it: work the section first, then come back. Commit to an answer before you run or reveal. The prediction is where the learning happens. Answers are hidden behind ▸ toggles.
Every example is offline: the default
PROVIDER=mockruns the whole repo with no key and no cost.
Section 2: The bare loop and its gaps
Recall. The loop in examples/01_bare_loop_recap.py works. Name the five things
it can't do that a harness adds.
▸ Answer
Observe (only print, no event stream), gate (every tool runs
unconditionally), intercept (no place to block a call or redact a result),
contain (no boundary on where a tool acts), and delegate (one loop, one
context: no subagents). Each is a seam the harness adds around the same loop.
Section 3: The harness and its event stream
Predict (02). You hand the same task to a Harness and iterate run(). Where
did the while loop go, and what do you write instead?
▸ Answer
The loop moved inside Harness.run(). You don't write a loop at all; you iterate
the event stream it yields (RunStarted, ModelTurn, ToolFinished, RunFinished,
...). That inversion is the whole value: every event is a seam to observe, react, or
record.
Section 4: Hooks
Predict (03). A pre-tool hook raises HookBlock on credential files; a
post-tool hook redacts API keys. For a safe file that happens to contain a key,
which hook fires and what does the model end up seeing?
▸ Answer
The post-tool hook fires (the pre-tool block only triggers on credential-looking paths, and this file's path is safe). The tool reads the file, then the post-tool hook redacts the key from the result, so the model summarizes the file but never sees the raw secret. Blocking (pre) and transforming (post) are different jobs at different points in the cycle.
Recall. Why put redaction in a hook instead of inside the read_file tool?
▸ Answer
Because it's a cross-cutting rule, not one tool's job. You want it applied to every tool's output, and you want to add or change it without editing tools. The harness seam is the one place it lives; this is how real systems ship guardrails (the Prompt Injection dive's defenses) without every tool re-implementing them.
Section 5: Permission policies
Predict (04). The policy is ask for write_file and deny for
run_command, and the demo auto-approves. What happens to each of the two tasks?
▸ Answer
The write_file is gated by ask, the callback approves, and it runs. The
run_command hits deny and is refused before executing, regardless of
approval; deny isn't something a human can wave through. The denial comes back as
a tool result, so the agent sees it and adapts.
Recall. What did lifting the policy out of the loop into its own object buy you?
▸ Answer
You can read it, unit-test it, diff it in code review, and swap it per environment
(strict in prod, loose in a dev sandbox), all without touching agent code. In the
Agents dive the policy was tangled into an approve callback threaded through the
loop; here it's a declarative object, the same shape as Claude Agent SDK permission
modes.
Section 6: The sandbox
Predict (05). The agent tries read ../../../../etc/passwd. Does a
startswith(root) check on the raw path string catch it? What does the sandbox do?
▸ Answer
A raw-string check can be fooled (the path may not literally start with the root, or
may use symlinks). The sandbox resolves the path first (os.path.realpath,
collapsing .. and following symlinks) and checks the canonical location against
the root, so the traversal is refused. The refusal comes back as a tool result, not
a crash.
Recall. Why an allowlist for commands instead of a blocklist?
▸ Answer
A blocklist ("reject rm") loses: there are infinite dangerous phrasings and
encodings. An allowlist names the few commands you trust and refuses everything else
by default. Reject-by-default is the only boundary that holds against an adversary
choosing the input.
Section 7: Subagents
Predict (06). The orchestrator has only the calculator; the research
subagent owns the knowledge-base tool. After delegation, what enters the
orchestrator's context: the subagent's search step, or just its answer?
▸ Answer
Just the answer. The subagent runs in a nested harness with its own context window; only its final result returns to the orchestrator. That's context isolation: the orchestrator's window never fills with the subagent's intermediate tool calls. (They share the sandbox/filesystem, but not the conversation.)
Section 8: Headless automation
Recall (07). What makes the harness suitable for a cron job or CI step, with
no human present?
▸ Answer
run_to_completion drives it to a final answer, and because everything is events
you fold the run into a machine-readable record (tools run, anything blocked, the
answer) as it happens. The only thing it emits is structured output another program
consumes: a log line, a webhook payload, or something CI can assert on (e.g. fail
the build if a tool was blocked).
Section 9: Computer use & hosted sandboxes
Recall (08). How is computer use the same loop you already know, and what's
different?
▸ Answer
Same observe → act → observe loop. What's different is the tool surface: the tools
are screenshot / click / type, and the observation fed back each step is an
image of a screen instead of a text tool result. A harness still adds the same
permission and hook seams, and a hosted sandbox (a provider-run VM/browser) so
the agent drives an isolated machine, not your own.
Recall. When should you reach for computer use, and when not?
▸ Answer
Reach for it only when the task lives in a GUI with no API to call. If a real tool or API exists, use that. Driving pixels is slower, costlier, and far less reliable than a typed tool call. Computer use is the fallback for the un-automatable, not the default.
Section 10: Durable runs (checkpoint & resume)
Predict (09). A two-step run (read a file, then compute) crashes right after
the first tool finishes. A fresh process resumes it. Does the resumed process re-run
the read? Why or why not?
▸ Answer
No. The completed read's result was checkpointed into the persisted transcript before the crash. On resume, the harness reloads that transcript and keeps looping the model sees the read result already present and moves straight to the calculation. The example's counter proves each tool ran exactly once across both processes. That's the whole mechanism: the transcript is the checkpoint.
Recall. Why is persisting the transcript enough to make a run resumable, when you don't separately save "which step I'm on"?
▸ Answer
Because the loop is deterministic given the transcript: every tool result is already fed back into it, so "where I am" is implied by "what's in the transcript." Reload it and the next model turn naturally produces the next step. Real durable-execution engines (LangGraph, Temporal) persist more, but the principle is the same: save the state the loop reads, and resuming is just continuing.
Section 11: Durable task state (run records)
Recall (10). Three runs finish in three states: done, failed, and stuck in
running. What does each mean, and what do you do with the running one?
▸ Answer
done = finished with an answer. failed = gave up (here, hit the step limit)
needs a fix or a bigger budget. Stuck in running = the process crashed mid-run
(the status was never advanced to done). You resume it from its checkpoint, which
the example does straight from the run log, knowing nothing but its id.
Recall. The checkpoint file powers two different features. What are they?
▸ Answer
Resuming one run (§10) and monitoring all runs (§11): a queryable task-state log of what finished, what's running, and what crashed. Same persisted state, two uses. That status column is the difference between an agent you hope finished and one you can prove did: the durable task state real agent systems are built on.
Section 12: Parallel subagents
Predict (11). Three independent research workers, each taking ~0.4s, run via
fan_out. Serially the batch takes ~1.2s. Roughly how long does the concurrent run
take, and why?
▸ Answer
About 0.4s: the time of the slowest worker, not the sum. The workers are independent, so there's no reason to wait between them; run them concurrently and the batch cost is the MAX, not the SUM (~3× here). Each still gets its own harness and context window (isolation); they only share the sandbox, which is why parallel workers should be independent and read-mostly.
Section 13: Steering a running agent
Predict (12). Mid-run, the operator injects "now compute 30 * 12" and then
interrupts. What does the agent do with the injected message, and does the interrupt
kill it mid-tool?
▸ Answer
The injected message becomes the newest user turn, so it steers the next step
the agent switches from the lookup to the calculation, no restart. The interrupt does
not kill it mid-tool: the harness only checks the controller at a step boundary,
so the current step finishes and the run halts cleanly. An interrupted run is
checkpointed as interrupted (resumable), not lost.
Recall. How does steering differ from the permission policy (§5)?
▸ Answer
The permission policy decides whether a tool runs before it runs (a synchronous gate). Steering acts on a run while it's in flight: inject a new instruction, queue follow-ups, or interrupt. Gate = "may this happen?"; steer = "here's a change of plan / stop."
Section 14: Orchestration as a graph
Recall (13). A graph classifies a ticket, routes it to a handler, runs a review
gate, and loops back to revise until it passes. Which two control-flow features does
that use that a straight sequence of steps doesn't?
▸ Answer
Branching (the router sends a billing ticket and a technical ticket to different
handler nodes, since the path depends on the state) and a cycle (the review gate routes
back through revise until the draft passes). Nodes + conditional edges give you both;
a straight pipeline gives you neither.
Recall. When do you build a graph vs. run the agent loop?
▸ Answer
If you can draw the flowchart, build a graph. Code drives the path, which is cheaper, predictable, and testable. Reach for the model-driven loop only when the path genuinely can't be known up front. Real systems mix them: a graph whose individual nodes each run a Harness (an agent as one step).
Capstone: agent_harness.py
Do. Run python hands_on/agent_harness.py "write file todo.txt containing: ship it" --yes, then run the same task without --yes. What changes, and does --yes
let the agent run a denied tool?
▸ Answer
With --yes the ask verdict on write_file is auto-approved; without it you're
prompted. Neither lets a denied tool run: --yes only auto-answers ask, and
deny (like run_command) is refused regardless. Allow/ask/deny are three distinct
verdicts, and only ask is a question.
Stretch. Add a second subagent, or tighten the policy to deny write_file
outright, and watch the event trace change. When the harness routes to a capability
you added just by registering it, the "configure, don't code the loop" idea has
landed.
Where to take it next
Point the harness at a real model (PROVIDER=openai or claude, add a key) and
give it a multi-step task. Watch the same hooks, policy, and sandbox govern a real
model's tool calls. The machinery didn't change, only who's choosing the tools.
Then open the Claude Agent SDK or Managed Agents docs: everything there will read as
the hardened, hosted version of the seams you just built.