#Exercises: make the learning stick

The API is stateless. "Memory" is just the message list you resend, and that list has a hard token ceiling (the context window). Every kept turn costs tokens (you re-send and re-pay for it each request) and pushes you toward overflow (the request fails, or quality drops). So you must choose what to keep.

It can't recall it. Turn 1 scrolled off the back of the window, so it isn't in the messages sent. For the model the conversation is the window; delete the introduction and the name is genuinely gone. Bounded and cheap, but forgetful.

Yes: it recalls the name and the thing it was asked to remember. Compaction replaced the oldest turns with a short summary (which preserves the facts) and kept the recent turns verbatim. What's thrown away is the exact wording of old turns, and you pay one summarization call. The risk: a bad summary drops a detail.

More compactions fire (the window fills sooner), but the name survives, because each compaction folds the prior summary into the next, and the canonical fact lines carry through. That's the point: bounded memory that doesn't forget the essentials.

A summary lives in that conversation's window. Close the session and it's gone. Long-term memory writes durable facts to a store outside any window and retrieves the relevant few back in when a new turn needs them. It's RAG pointed at the conversation: persist, then retrieve by relevance.

The window holds nothing from session one, so without retrieval the fact simply isn't present and the model says it doesn't have it. With long-term memory, the relevant fact ("launch is Friday") is recalled from the store and injected into the system prompt, back in the window where the model can use it.

Models attend most reliably to the start and end of a long context and are worse at using material buried in the middle. order_for_recall places the highest-priority sections at the edges and the filler in the middle: same tokens, better odds the key chunk is actually used.

The highest-priority sections that fit survive: the system prompt and the user's question are non-negotiable; long-term memory and the on-topic retrieved doc make it; the marketing blog (lowest priority) is cut first. Prioritize, then pack: a giant low-priority doc must never crowd out a small essential one.

Both halves of context rot. Tokens: the bloated window costs ~10× the tokens (so ~10× the spend and more latency), on every turn. Quality: it returns the WRONG name: the buried distractor won. On the mock that flip is deterministic (it takes the last "my name is ..." it sees); on a real model the same rot is subtler, with padding dilutes the signal and pulls attention to a plausible irrelevant passage. Relevance beats volume.

Each agent step appends a tool call and its full (often huge) result, and most of those results are spent after one step. Pruning keeps the agent's reasoning and the most recent observations verbatim, and stubs/drops the old ones: a big token cut with the train of thought intact.

Not necessarily. Here it costs about 1.5× more. Providers cache the prompt prefix, and any change to the prefix invalidates everything after it. Append-only never changes its prefix, so almost every token is a cheap cache read (~0.1×). Compaction rewrites the system prompt (the summary) and drops old turns, a prefix change, so those turns are cache misses billed at full write price (~1.25×) on the whole context. Fewer tokens, bigger bill.

You still need it eventually: an append-only window grows without bound, so on a very long conversation its cheap-but-huge reads finally cost more than a small uncached context. The two axes ("fewer tokens" vs "cheaper bill") cross over. The fix is cadence: compact rarely and in bulk so you pay the cache miss once instead of every turn, keep the stable part of the prefix byte-identical, and watch cache_read_input_tokens vs cache_creation_input_tokens to see which regime you're in.

The first run persisted the durable facts to .ctx_memory.json. The second run is a fresh process with an empty window, but it loads that file and recalls the launch fact into its system prompt. In-session memory (compaction) and cross-session memory (the store) are different layers; the capstone uses both.

Compactions climb each time the budget is exceeded, and the token line stays at or under 200, because compaction folds old turns into the summary the moment they'd push you over. Bounded memory: the window can't grow without bound no matter how long you talk.

Better: it summarizes prose far more intelligently and recalls paraphrased facts the mock's keyword match would miss. Worse: it's nondeterministic, and a summary can quietly drop a detail, and recall can miss. That's exactly why memory quality is something you measure (the Evals dive), not assume.