Core path - 2 of 8

#Claude API: A Guided Deep Dive

A hands-on playground for learning the Claude (Anthropic) API from zero. You'll build a real CLI tool that answers questions about your code, and along the way you'll understand every moving part. Messages and content blocks, the system prompt, the sampling knobs (temperature, top_p, max_tokens, stop), token counting, cost, and the two things that make Claude feel like Claude: tool use and extended thinking.

Walk through this repo rather than reading it. Each section ends with something to run. Do the running. That's where the learning is. And once a section clicks, EXERCISES.md has a quick predict-then-run prompt for it. Committing to an answer before you run is what makes it stick.

#0. The one big idea

The Claude API is simpler than it looks.

You send a list of messages. You get back a message.

That's it. Everything else, the system prompt, the knobs, the token math, is detail on top of that one request and response. Hold onto that and nothing below will feel complicated.

One twist is worth knowing up front. The message you get back isn't a plain string. It's a list of content blocks, each one tagged with a .type. For a normal answer you read the text blocks. That same list is how Claude later hands you its reasoning and its tool requests, so it pays to meet it early.

#1. Setup (5 minutes)

# 1. Create an isolated Python environment
python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate

# 2. Install dependencies
pip install -r requirements.txt

# 3. Set up your API key (it does NOT go in .env)
cp .env.example .env               # optional; holds no secrets
#    Store your key in your OS keychain and run scripts with `secrun`: 2-minute
#    setup in ../docs/SECRETS.md. Get a key: https://console.anthropic.com/settings/keys

# 4. Confirm everything is wired up correctly (makes no API call, costs nothing)
secrun python check_setup.py

check_setup.py is your first stop if anything goes wrong. It checks your Python version, your installed packages, and your key, and tells you exactly what to fix. Green across the board means you're ready for Section 2.

You'll need a key for everything here, but the token-counting parts are free. Claude has no offline tokenizer, unlike some APIs, so counting tokens is an API call that's free and unbilled. Sections 5 and 6 need a key and never cost you anything. More on that below.

One optional extra key. The embeddings example (Section 8) uses Voyage AI, Anthropic's recommended embeddings provider, which has its own VOYAGE_API_KEY. Everything else uses only your ANTHROPIC_API_KEY. Both go in .env; the Voyage one is optional.

#2. Your first request

secrun python examples/01_basic_chat.py

Open examples/01_basic_chat.py and read it. It's tiny. The shape of every call you'll ever make is right there:

response = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "In one sentence, what is an API?"}],
)
for block in response.content:
    if block.type == "text":
        print(block.text)

Five things to internalize:

#3. The system prompt, and roles

A conversation is a transcript, and every message carries a role. On Claude there are two roles inside the list.

• user is what the human says. The first message has to be a user message.
• assistant is what the model said. You re-send past assistant messages to give the model memory. The API itself is stateless, so the conversation exists only in the list you send each time.

The third piece is the system prompt, which holds the standing instructions, the persona, and the tone. It isn't a message at all. It's a separate top-level system= parameter. Set it once and it steers everything, which makes it the strongest lever you have.

secrun python examples/02_roles.py

Experiment: open examples/02_roles.py, change the system prompt to "You are a grumpy pirate.", and rerun. Same question, completely different voice.

#4. The knobs that shape a response

These parameters control how the model answers. Each one has its own runnable example.

#temperature, or how bold the word choices are

0.0 is focused and repeatable, 1.0 is the most varied and also Claude's default. Claude's range is 0.0 to 1.0 rather than 0 to 2. For code and facts, go low.

secrun python examples/03_temperature.py

#max_tokens, a hard cap on the answer's length

This caps output tokens and not input, and on Claude it's required on every request. When the budget runs out the model gets cut off, possibly mid-sentence. Watch stop_reason. "max_tokens" means truncated and "end_turn" means it finished on its own.

secrun python examples/04_max_tokens.py

#top_p, or how many options the model may consider

Nucleus sampling. 0.1 allows only the most obvious tokens, 1.0 allows everything and is the default. Tune temperature or top_p, never both. They interact in confusing ways.

secrun python examples/05_top_p.py

#stop_sequences, to halt generation at a marker

A list of strings. Generation ends the moment one of them would appear, and the stop text itself isn't included. Good for cutting lists short or stopping at a delimiter.

secrun python examples/06_stop_sequences.py

Quick reference:

Where this is going. Anthropic's newest models, Claude Opus 4.8 and Claude Fable 5, have removed the sampling knobs temperature, top_p, and top_k entirely. Sending them errors. Those models steer through prompting plus the effort and thinking controls in Section 8 instead. The knobs above still work on the workhorse models and are worth understanding, but the newest models have moved past them.

#5. Tokens: what you actually pay for

Models don't read characters or words. They read tokens, which are chunks of text, often word fragments. Rough rule: 1 token is about 4 English characters, roughly three quarters of a word. Rough isn't good enough for budgeting, so we count exactly.

Here's a real Claude difference. There's no offline tokenizer. You count tokens by asking the API, through client.messages.count_tokens(...). The good news is that it's free, unbilled, uses none of your output budget, and exact. The trade is that it needs your key and a network call, and you get back a count rather than the individual token pieces, because Anthropic's tokenizer isn't public.

secrun python utils/tokens.py          # count a sentence's tokens (free API call)

Counting matters for three reasons.

1. Cost. You're billed per token, as the next section covers.
2. Limits. Every model has a maximum context window covering input and output. Overflow it and the request fails.
3. Intuition. Watching the count change as you edit a prompt teaches you how the model reads your text.

See utils/tokens.py for count_tokens() (a raw string) and count_message_tokens() (a full request, including the system prompt, which counts toward your input tokens too).

#6. Cost estimation

Anthropic charges separately for input and output tokens, and output costs several times more. utils/pricing.py holds a small price table and an estimate_cost() helper. Counting is an API call, but the cost math is pure local computation with no network and no key.

secrun python examples/07_token_counting.py   # tokens -> dollars, across models

The sample output shows the same request costing wildly different amounts depending on the model, which is why choosing the right model is part of prompt engineering.

Prices change. The table in utils/pricing.py is a snapshot, so always confirm at https://platform.claude.com/docs/en/about-claude/pricing.

#7. The first capstone: ask.py

Everything above comes together in one tool. Ask a question about a code file, see the token count and estimated cost before you spend anything, get the answer, then see the actual usage and cost after.

# See the size and cost first: the counting call is free, so no money spent:
secrun python hands_on/ask.py snippets/buggy.py "Is there a bug here?" --dry-run

# For real:
secrun python hands_on/ask.py snippets/buggy.py "Is there a bug here?"

# Now turn the knobs you just learned:
secrun python hands_on/ask.py snippets/buggy.py "Rewrite this cleanly" --temperature 0
secrun python hands_on/ask.py snippets/buggy.py "List the issues" --max-tokens 200 --stop "4."
secrun python hands_on/ask.py snippets/buggy.py "Explain this" --model claude-sonnet-4-6

Run secrun python hands_on/ask.py --help to see every knob explained inline. Read the source in hands_on/ask.py. It's commented as a tutorial, especially build_messages(), which assembles the request with the system prompt kept separate, and the usage and cost reporting at the end.

Suggested exercise: point hands_on/ask.py at your own code, try the same question at --temperature 0 vs --temperature 1, and watch both the answers and the cost change.

#8. Beyond the basics

With the core down, here are the most useful next capabilities. Each one is a runnable example in the same numbered style, and every one is still a variation on "send messages, get a message."

#Streaming, to get the answer as it's typed

client.messages.stream(...) delivers the response in small pieces as it's generated, so the user sees text appear immediately. It's also the recommended way to do long generations, since it dodges request timeouts.

secrun python examples/08_streaming.py

#Structured outputs, to make the model return real JSON

Constrain the reply to match an exact JSON Schema you define, via output_config={"format": ...}. The end of fragile "please reply in JSON" prompting. client.messages.parse() with a Pydantic model is the friendlier version.

secrun python examples/09_structured_outputs.py

#Tool use, to let the model use your code

You describe tools. The model decides when to call one and with what arguments. You run it and feed the result back. This is how a model gets to actually do things, like query a database or hit an API.

secrun python examples/10_tool_use.py

#Extended thinking and effort, letting the model reason first

This is Claude's signature capability. On hard problems, let it think in dedicated thinking blocks before answering, and dial how much effort it spends. On the newest models this replaces the temperature and top_p knobs.

secrun python examples/11_thinking.py

#Embeddings via Voyage AI, turning text into vectors for search and similarity

Embeddings convert text into numbers that capture meaning, so you can rank text by similarity. That's what semantic search and RAG are built on. Anthropic has no first-party embeddings endpoint and recommends Voyage AI, a separate provider with its own SDK (voyageai) and key (VOYAGE_API_KEY). So a real Claude app looks like this: Claude reasons, Voyage embeds. The example ranks sentences against a query, including one that shares no words with it at all.

secrun python examples/12_embeddings.py

#Multi-turn conversations, because the API has no memory

Each request is stateless and Claude remembers nothing. A chatbot that "remembers" is you re-sending the whole messages list every turn, appending each new user and assistant message, with the system prompt staying separate. The example is a tiny REPL that grows that list.

secrun python examples/13_conversation.py

#Error handling and retries, for surviving the real world

The network blips, you hit a rate limit, the service is briefly overloaded with a 529. The SDK already retries transient failures (429, 5xx, connection) with backoff. Your job is to tune timeout and max_retries and catch the typed exceptions, so "fix your request" errors get handled differently from "try again later" ones.

secrun python examples/14_error_handling.py

#Pydantic validation, for typed and validated responses

Instead of a hand-written JSON Schema plus json.loads into an untyped dict, define your shape as a Pydantic model and pass it as output_format. The SDK sends the schema, constrains Claude, and hands back a validated instance on .parsed_output, with typed attributes, enforced constraints, and editor autocomplete.

secrun python examples/15_pydantic_validation.py

#Formatting output as Markdown, tables, and code blocks

Claude answers in Markdown, and dumped raw to a terminal that is a mess of literal **asterisks**. The rich library renders Markdown, syntax-highlighted code, and real tables in the terminal. That's the difference between output you skim and output you squint at.

secrun python examples/16_rich_output.py

#Server-Sent Events (SSE), the protocol under streaming

Every streaming Claude response travels over SSE, a plain HTTP response that stays open and drips event: <type> and data: <json> pairs until the server is done. Claude's event stream is richer than most, with message_start, content_block_delta, message_delta, and more. This example shows all of them at the raw level, plus per-token timing and partial response accumulation.

secrun python examples/17_sse.py

#Vision, sending images alongside text

Claude is multimodal. The user content becomes a list of blocks, a text block plus an image block, where the image is either a URL Claude fetches or a local file sent as base64 with a media_type. Images are billed as input tokens, scaled by pixel size. Claude reads images and doesn't generate them.

secrun python examples/18_vision.py            # or: secrun python examples/18_vision.py my_image.png

#The Batch API, half price for non-urgent work

For work that isn't interactive, like classifying 10k reviews or summarizing a backlog, submit many Requests at once and get results within 24 hours, usually under one, at 50% off. Poll processing_status until "ended", then stream the results keyed by custom_id, since they come back in any order.

secrun python examples/19_batch_api.py

#Prompt caching, so you don't re-pay for a repeated prefix

Mark a stable block with cache_control={"type": "ephemeral"}. The first request writes it at about 1.25×, and later requests with an identical prefix read it at about 0.1× and faster. It's a prefix match, so keep the constant part first (system prompt, tools, a document) and the question last. The example shows cache_read_input_tokens kicking in.

secrun python examples/20_prompt_caching.py

#Async and concurrency, for many requests at once

A single call is mostly idle waiting on the network, so independent prompts should run concurrently. AsyncAnthropic + asyncio.gather + a Semaphore (bounded concurrency) finishes a batch in roughly the time of the slowest call, while staying under your rate limit. The example times sequential vs. concurrent.

secrun python examples/21_async_concurrency.py

#9. The second capstone: extract.py

Where ask.py returns prose, extract.py returns data. Point it at messy free-form text and it pulls out a clean, typed, validated structure, then shows it as a Markdown summary and a real table. This is where examples 15 (Pydantic) and 16 (rich) finally do something useful on a realistic task.

# See tokens + cost first: the counting call is free:
secrun python hands_on/extract.py snippets/meeting_notes.txt --dry-run

# Extract action items (owner, due date, inferred priority) into a table:
secrun python hands_on/extract.py snippets/meeting_notes.txt

# Want the raw validated JSON instead? (e.g. to pipe into another tool)
secrun python hands_on/extract.py snippets/meeting_notes.txt --json

Read the source in hands_on/extract.py. The Extraction and ActionItem Pydantic models are the schema Claude must follow, and render() is the rich table. Suggested exercise: point it at your own meeting notes or an email, or change the models to extract something else entirely, like contacts or invoice line items. The prompt barely changes.

#10. The third capstone: streaming_server.py

Where ask.py and extract.py are CLI tools, streaming_server.py is a web service. A FastAPI backend that streams Claude responses to a browser over SSE, and it shows three production concerns: token-by-token forwarding, client disconnect detection, and error recovery with retries.

# Start the server (auto-reloads on file saves):
uvicorn hands_on.streaming_server:app --reload

# Then open: http://localhost:8000

Open the browser's Network tab and click the /stream request to see the raw text/event-stream response. Close the tab mid-stream to watch the server log "client disconnected" and stop the Claude call. Read the source in hands_on/streaming_server.py. The three-phase generator _stream_tokens is the pattern every production streaming endpoint follows.

Suggested exercise: ask something that generates a long response and close the browser tab mid-stream. The server logs client disconnected after N chunks aborting Claude call and the Claude API call stops immediately, with no wasted tokens.

#11. The fourth capstone: rag.py

The embeddings example in Section 8 ranked sentences by similarity. rag.py puts that to work. It answers questions over a small knowledge base by retrieving the most relevant facts and pasting them into the prompt, which is the smallest thing you could still call retrieval-augmented generation (RAG). No vector database, no framework. Just the embeddings and chat calls you already know, wired together from scratch.

Hold onto one idea. A model can only answer from what's in its context window, and RAG decides what to put there.

# Answer the built-in demo question from the knowledge base:
secrun python hands_on/rag.py

# Ask your own:
secrun python hands_on/rag.py "Can I get a refund?"

# The contrast that makes the point: the same question with NO retrieved context:
secrun python hands_on/rag.py "How long are deleted notes kept?" --no-rag

# See exactly what gets retrieved and what prompt gets sent:
secrun python hands_on/rag.py "What plans are there?" -k 5 --show-prompt

The knowledge base describes a made-up app, so Claude can't fall back on training and a correct answer can only come from retrieval. Run it with --no-rag and watch the model guess or refuse. That contrast is the lesson. It also shows the real shape of a Claude app, where Voyage embeds and Claude reasons, so it uses both keys. --no-rag needs only your Anthropic key.

Read the source in hands_on/rag.py. retrieve() is the whole embed, score, rank loop, and build_user_message() is the entire augment step. RAG is mostly good string assembly. Suggested exercise: add a fact to KNOWLEDGE_BASE, then ask a question only that fact can answer.

#Where to go next

You've now covered the essentials, the most common extensions, and four capstone projects. Further on:

• Prompt caching. Cache a large, repeated prefix so you pay about 0.1× for it on later requests, up to 90% cheaper. The single biggest cost lever for context-heavy apps, and a natural follow-on to this repo's cost theme.
• Vision. Pass images, and PDFs, to Claude as content blocks alongside text.
• Citations. Have Claude cite the exact source spans it used from documents you provide.
• The Batch API. Submit many requests asynchronously at 50% off, for work that is not latency-sensitive.
• Agents and MCP. Multi-step tool-using loops, and connecting Claude to external tools through the Model Context Protocol.
• Local and self-hosted models. Claude is hosted-only, but the API skill transfers. Open-weight models like Llama and Mistral run on your own hardware for privacy or cost, and local runtimes like Ollama speak an OpenAI-compatible API, so the same client code reaches them. See the OpenAI repo's local-serving example. Same "send messages, get a message" shape, a different base_url.
• RAG at scale. The rag.py capstone re-embeds a handful of facts on every run. Real systems embed once into a vector database, chunk long documents, and add reranking and evaluation. Enough moving parts to deserve a deep dive of its own.

Every one of these sits on top of the "send messages, get a message" idea you started with.

#Troubleshooting

Hit a snag? Run secrun python check_setup.py first. It catches most problems. The rest, sorted by the error you see:

Still stuck? Every example is small and self-contained. Open the file, read the docstring at the top, and run it directly. The error message almost always points at the line.

#From teaching code to production

Every example here takes shortcuts that are perfect for learning and wrong for a real deployment. Here's the map from each shortcut to what production uses.

All seven concerns (observability, cost, reliability, caching, guardrails, prompt versioning, and eval gates) get built from scratch and wired into one running app in Production, which is #8 in the series. It runs offline on a mock provider, so you can see the whole ops machinery with no key and no cost.

#File map

check_setup.py              ← run first: verifies Python, packages, and your key
EXERCISES.md                ← active-recall prompts, one per README section
hands_on/
  ask.py                    ← capstone CLI: ask a question about a code file
  extract.py                ← capstone CLI: extract validated data from free text
  streaming_server.py       ← capstone server: stream AI responses over SSE
  rag.py                    ← capstone CLI: answer questions over a knowledge base (RAG)
  static/index.html         ← browser UI for the streaming server
utils/
  tokens.py                 ← token counting via the free count_tokens API
  pricing.py                ← price table + cost estimation
snippets/buggy.py           ← a sample file to ask questions about
snippets/meeting_notes.txt  ← sample free-form text for extract.py
examples/
  01_basic_chat.py          ← the minimal request + content blocks
  02_roles.py               ← the system prompt, user / assistant
  03_temperature.py         ← randomness
  04_max_tokens.py          ← length cap + stop_reason
  05_top_p.py               ← nucleus sampling
  06_stop_sequences.py      ← halting generation
  07_token_counting.py      ← tokens & cost
  08_streaming.py           ← stream the answer as it's generated
  09_structured_outputs.py  ← guaranteed schema-conformant JSON
  10_tool_use.py            ← let the model call your functions
  11_thinking.py            ← extended thinking & effort
  12_embeddings.py          ← vectors & semantic similarity (via Voyage AI)
  13_conversation.py        ← multi-turn chat & the stateless API
  14_error_handling.py      ← timeouts, retries & typed exceptions
  15_pydantic_validation.py ← typed, validated responses via Pydantic
  16_rich_output.py         ← Markdown, tables & code blocks in the terminal
  17_sse.py                 ← SSE protocol: raw events, timing, partial accumulation
  18_vision.py              ← send an image (URL or local base64) alongside text
  19_batch_api.py           ← submit many requests at 50% off, results within 24h
  20_prompt_caching.py      ← cache_control: cache a long repeated prefix at ~0.1x
  21_async_concurrency.py   ← AsyncAnthropic + asyncio.gather + a Semaphore (throughput)

#Footnote: quieting Pylance/type-checker noise

Two patterns trip the type checker repeatedly with the Anthropic SDK. Pre-empt them and new files stay clean:

1. Assigning messages / tools to a variable (rather than passing the literal straight into create()) makes Pylance infer a too-narrow type like list[dict[str, str]]. Annotate with the SDK's own param types; you also get key autocomplete:
   python

   from anthropic.types import MessageParam, ToolParam
   messages: list[MessageParam] = [...]
   tools: list[ToolParam] = [...]
   

2. response.content is a list of content blocks, not a string. There's no single .content string to read like a chat-completion's .content, even a plain text reply comes back as [TextBlock(type="text", text="...")]. Walk the list and pull out the "text"-typed blocks:
   python

   reply = "".join(block.text for block in response.content if block.type == "text")
   

The repo's .vscode/settings.json also sets python.analysis.typeCheckingMode to basic, which keeps the useful checks (undefined names, bad attrs/args) while dropping the strict dict-vs-TypedDict complaints.

#The series

This is one of the standalone, hands-on deep dives into building with LLM APIs. Eight core dives, plus the bonus ones listed below. Each one stands on its own, with its own setup, examples, and capstone, and they all share one house style. Provider-agnostic, built from scratch with no frameworks, offline-first examples, and a real capstone at the end. Do them in any order. This sequence builds naturally.

1. OpenAI API: the API from zero
2. Claude API: the same ideas, the Anthropic way
3. Prompt Engineering: shape model behavior with better prompts, using zero-shot and few-shot, chain-of-thought, and roles
4. RAG: answer questions over your own documents
5. Evals: measure whether a change actually helps
6. Agents: give a model tools and a loop so it can act
7. Prompt Injection & Guardrails: attack and defend all of the above
8. Production: operate one app end to end, across observability, cost, reliability, caching, guardrails, prompt versioning, and eval gates

Bonus dives, standalone and slotting in where they're most useful:

• Context Engineering: manage what's in the window, with memory, compaction, and assembly
• AI Data Engineering: the corpus behind the index, with versions, lineage, ACLs, and deletes
• Multimodal: images and audio as well as text
• Fine-tuning: teach a model new behavior by example
• MCP: serve tools, data, and prompts to any LLM over a standard protocol
• Local Models: run open-weight models on your own machine
• Agent Harnesses: build on the loop, adding hooks, permissions, sandboxing, and subagents
• Realtime Voice: low-latency speech-to-speech agents
• Observability: watch a running app over time, covering drift, quality, alerting, and the feedback loop
• Architecture: the seams between the components, each decision measured rather than asserted
• GenAI Security: treat the model as an untrusted principal, and put identity, supply chain, isolation, budgets, and release gates around it
• Inference Platform Engineering: turn finite GPU memory and a request queue into latency, throughput, and a fleet size you can defend
• Testing & Delivery: decide whether a build is fit to promote, using evidence, gates, staged rollout, and rollback
• Professional Tools: rebuild each hand-written piece with the tool professionals reach for, and measure both

And the whole series lands in one codebase in the capstone: a codebase Q&A tool built step by step, one tag per dive.

You're here: #2, Claude API.