av / dives /Agents: Exercises
source about me

Core path - 6 of 8

Exercises: make the learning stick

Reading code teaches you less than predicting what it will do and then checking. This file turns each section of the README into a few quick active-recall prompts.

How to use it: work the section first, then come back. Commit to an answer before you run or reveal. The prediction is where the learning happens. Answers are hidden behind ▸ toggles.

Examples 01 and 10 are (offline): no API call, no cost. The rest make small, cheap calls.


Section 2: What a tool is (offline)

Recall. When the model "uses a tool," does it run your function? What does it actually do?

▸ Answer

No: the model never runs anything. It emits a request: a tool name plus arguments. Your code decides whether and how to execute it. That gap is where all of an agent's safety lives (approval, sandboxing, validation).

Do (offline). In examples/01_tools.py, the model only ever sees a tool's name, description, and parameter schema, not the function body. Why does that make the description a piece of prompt engineering?

▸ Answer

Because the description is the model's only basis for deciding when and how to call the tool. A vague description ("does stuff") leads to misuse; a precise one ("use this for arithmetic; expression like '2+2'") leads to correct calls. You're programming the model's behavior through that text.


Section 3: One tool call

Predict, then run. In examples/02_one_tool_call.py, you ask "What is 23 * 47?" with the calculator available. Will the model reply with the number, or with something else?

▸ Answer

With a tool call request (calculator, expression="23 * 47"), not the number. It defers the math to the tool. You then run it, and to turn that result into a final answer you'd feed it back and ask again, which is the loop.


Section 4: The agent loop

Recall. Describe the agent loop in one sentence. What makes it stop?

▸ Answer

Ask the model; if it requests tools, run them, append the results, and ask again; repeat until it responds with no tool calls; that final, tool-free response is the answer. (A step cap is the backstop in case it never stops.)

Do. In examples/03_agent_loop.py, give it a question needing two or three calculations and watch the trace. Does it make all the calls up front, or use each result to decide the next?

▸ Answer

Usually one (or a few) at a time, using earlier results to inform later steps that step-by-step, result-dependent reasoning is exactly what the loop enables and a single call can't do.


Section 5: Multiple tools

Predict. Give the agent calculator and search_notes and ask "what does the Plus plan cost per year?" Which tool runs first, and why?

▸ Answer

search_notes first (to find the monthly price, a product fact), then calculator (to multiply by 12). The model routes each sub-task to the tool whose description fits. Nothing hard-codes that order; it's the model's choice, guided by the tool descriptions.


Section 6: Limits & error recovery

Recall. Why feed a tool's error back to the model instead of crashing? And what does max_steps protect against?

▸ Answer

Feeding the error back lets the model see what went wrong and adapt (retry with different inputs, or explain): robustness instead of a dead program. max_steps protects against an infinite loop: a model that keeps calling tools and never finishes would otherwise run forever (and run up a bill).

Do. Set max_steps=1 on a task that clearly needs several steps. What does the agent return, and what does stopped_early tell you?

▸ Answer

It returns a "stopped early" message and stopped_early=True, because it hit the ceiling before reaching a final answer. That flag is how your code knows the result is incomplete rather than a real answer.


Section 7: Human-in-the-loop

Recall. What makes a tool require approval, and what happens to the agent when you deny one?

▸ Answer

You mark it dangerous=True; the loop then calls your approve callback before running it. A denial is returned to the model as a normal tool result ("permission denied"), so the agent acknowledges it and adapts instead of forcing the action.


Section 8: Observability

Recall. Why is a step trace essential for agents specifically, more than for a single LLM call?

▸ Answer

Because an agent makes its own multi-step decisions (which tools, in what order, with what arguments. When it goes wrong, the why is in that sequence. Without a trace you're debugging a black box; with one you can see (and later eval) exactly what it did.


Section 9: Memory

Predict, then run. In examples/08_memory.py, ask it to "search the plans," then ask "which is the cheapest paid one?" Does the second question work? Why?

▸ Answer

It works, because the same history list is passed back each turn, so the earlier search and its results are still in the conversation the model sees. The API is stateless; the memory is the growing list you choose to resend.


Section 10: Multi-agent

Recall. In examples/09_multi_agent.py, what is the research sub-agent, from the orchestrator's point of view?

▸ Answer

Just another tool. Calling research(question=...) looks identical to calling any tool, but its function body runs a second agent loop with its own prompt and tools. Sub-agents are agents wrapped as tools; that's how large systems decompose.


Capstone: agent_cli.py

Do. Run secrun python hands_on/agent_cli.py "Save a note titled 'todo' with body 'x'" and deny the approval prompt. Then run it again with --yes. Then add --trace to either. You've now exercised the loop, the approval gate, and observability in one tool.

Stretch. Add a new tool to agent/tools.py (say, a word_count(text) tool), include it in default_tools(), and ask the capstone to use it. Adding a capability is just: write a function, describe it, register it. That's the whole extensibility story.


Going further: three more agent patterns

Recall (11). You can solve a support task with a hard-coded workflow or with the agent loop. When should you not reach for an agent?

▸ Answer

When you can already draw the flowchart. If the steps are known, a workflow (classify → route → handle, orchestrated in code) is cheaper, more predictable, and easier to test. An agent earns its cost and unpredictability only when the path is genuinely open-ended.

Recall (12). Planning and reflection are both extra LLM passes around the loop. What does each buy you, and what's the weakness of self-reflection?

▸ Answer

Plan (before) keeps a multi-step task from dropping a sub-goal. Reflect (after) catches half-answers and slips, then revises. The weakness: a model grading itself is fallible; the strong version grounds the critic in a real check (tests, a schema, a verifier), as in the prompt-engineering "reflexion" lesson.

Predict (13). The model asks for three independent get_weather calls in one turn. Why does running them concurrently help, and roughly how much?

▸ Answer

Independent calls don't depend on each other, so there's no reason to wait. Run them concurrently and the turn finishes in the time of the slowest call, not the sum. Three ~0.8s calls: ~2.4s sequential vs ~0.8s parallel (~3×). The final answer is a normal completion, so you can stream it on top.

Recall (14). Example 13 streams the final answer; example 14 streams every turn. What has to change in the loop, and what's the one fiddly part of streaming a turn that also calls a tool?

▸ Answer

Almost nothing changes. You swap run_turn for stream_turn and print the text deltas via a callback; the rest of the loop (run tools, feed results back, repeat) is identical. The fiddly part is reassembling the tool call: OpenAI streams the arguments as JSON fragments you must concatenate by index before you can parse them. That lives in agent/providers.py so the loop stays clean.


Section: Provider-hosted tools

Predict (15). You run examples/15_hosted_tools.py with the hosted web-search tool declared. How many client-side tool rounds does your loop handle, and does that mean no search happened?

▸ Answer

Zero client-side rounds, and search did happen. A hosted tool is run by the provider inside the turn, not by your loop. You send one request and get one final answer already grounded in search; there's no tool_use/tool_result round-trip for your code, even though the provider ran search one or more times (the example prints that count). Client-executed and hosted tools are different in kind.

Recall. What do you give up by using a hosted tool instead of a client-executed one, and what do you gain?

▸ Answer

You give up control: the call runs on the provider's side, so Section 7's approval gate can't reach it, and you can't custom-log, validate, or sandbox it. You gain zero plumbing: no schema, no execution, no result round-trip. Real agents mix both: client tools for actions you must govern, hosted tools (web search, code execution) for capability you're happy to rent.


Bonus: MCP: a tool you didn't ship with (offline)

Predict (10). The client turns each remote tool descriptor into an ordinary Tool object and hands it to the loop. When the agent later "runs" one, what does that tool's func actually do, and could run_agent from example 03 tell the difference from a local tool?

▸ Answer

The func is a closure that sends a tools/call request over the protocol (JSON lines over the server's stdin/stdout) and reads the text content blocks back. The loop can't tell the difference; the converted tool has the same name, description, schema, and callable shape as a local one, so tracing and approval gates apply unchanged. That's the payoff of the conversion step: the transport is invisible to the loop.

Recall. Example 01 said a tool is "a name, a description, and a JSON Schema." What does MCP add to that picture, and what still never happens, whether the tool is local or served across the world?

▸ Answer

MCP adds discovery and a wire format: tools/list advertises those same three things from another process (another team, another language, another machine), and tools/call invokes one by name with JSON arguments, with no bespoke glue per tool. What never changes: the model still never runs anything. It only ever sees the menu entry and emits a request. Now even your client doesn't hold the implementation; the server does. (For the protocol's other primitives, like resources, prompts, HTTP transport, and security, see the MCP deep dive.)


Where to take it next

Invent your own. Wire one of your real functions in as a tool, something that reads a file, hits an API you own, or queries a database. Mark it dangerous if it writes, and let the agent drive it. The first time an agent completes a multi-step task you didn't script step-by-step, the idea has landed.