#Exercises: make the learning stick

A list. A single request can return several alternative completions (via the n parameter), so the replies live in a list and [0] takes the first. With the default n=1 there's exactly one.

It doesn't. You do. Every turn you resend the entire messages list, including the earlier user and assistant turns. The "memory" is just that growing list being sent each time. (You'll build exactly this in examples/12_conversation.py.)

At 0 they'll be nearly identical every time (focused, near-deterministic, though never a 100% guarantee). At 1.5 they'll diverge, sometimes wildly. This is the whole point of the knob: low for facts/code, high for variety.

"length": the model was cut off by your cap, not because it was done. A natural finish shows "stop". Watching finish_reason is how you detect truncated answers in real code.

They both reshape the same probability distribution the model samples from, so their effects interact in ways that are hard to reason about. Pick one lever and leave the other at its default.

"unbelievable" splits into multiple sub-word tokens; common short words are often one. Whitespace is not free: runs of spaces and newlines are tokens too. Seeing the Pieces: list is the fastest way to build intuition for how the model "sees" text.

Far more on gpt-4o. Do the arithmetic with estimate_cost() to see the exact multiple: then notice that output tokens dominate, since output is priced several times higher than input. Choosing the cheaper model for a task is real money saved.

The input estimate should be near-exact; the gap is on the output side, because the dry run can only assume an output length. The real answer might be shorter or longer than the assumed ~500 tokens, moving the actual cost accordingly.

No. The model only emits the function name and arguments. You run the function and feed the result back in a follow-up message. The model never executes your code. It only asks for it to be run.

Yes, and that's the entire value of embeddings. They capture meaning, not word overlap, so "the feline napped" can rank near "a cat is sleeping." This is why embeddings power semantic search and RAG.

Time-to-first-token. The user starts reading immediately instead of staring at a blank screen until the whole answer is ready. Same total time, far better perceived responsiveness, which is why chat UIs stream.

A list of parts, a text part and an image_url part, in one user message. The image is either a URL the model fetches, or a local file inlined as a base64 data: URI. The image is billed as tokens, scaled by its pixel size.

While a model is reasoning it doesn't take sampling knobs; luna rejects temperature at any effort above "none". You steer how hard it thinks with reasoning_effort. The reasoning_tokens are the model's hidden chain of thought, generated before the visible answer and never shown to you, but still billed. Note the default: luna starts at "medium", so a request that doesn't ask still reasons. That's why every other example sends "none". The GPT-5.6 tiers default to "none" instead, so check per model.

You trade immediacy: results land within 24h instead of instantly. Each line's custom_id is echoed in the results file, so you can match every answer back to the request that produced it.

Caching only helps the prefix that's byte-for-byte identical. Put the constant block (system prompt, tool catalog, document) at the front and the variable question at the back, and the long prefix is served from cache at a discount on the second call (cached_tokens).

Each request is mostly idle network waiting, so overlapping them finishes in about the time of the slowest call. The Semaphore caps how many run at once, so you get the speedup without blowing past your account's rate limit.

No. It's a separate, free classifier returning category flags + scores. Call it on user input on the way in and model output on the way out, refusing or redacting when flagged is true.

A confident answer puts almost all probability on one token; an uncertain one spreads probability across alternatives. That spread is a usable confidence signal: auto-accept the confident ones, route the shaky ones to review.

At temperature=0 the model is already deterministic (always the most likely token), so a fixed seed wouldn't visibly be doing anything, and you couldn't tell its effect apart from temperature=0's own determinism. Running at temperature=0.9 keeps real randomness in play, so a matching seed across two calls visibly pins the output down, while an unset seed visibly doesn't. Either way, OpenAI can change the backend, and determinism isn't guaranteed across such changes; the system_fingerprint field is the tell: if it changes between calls, the backend shifted and identical inputs can drift even with the same seed.

output_text combines the text from message items and is the convenient path when text is all you need. output is heterogeneous. A response can include reasoning, tool-call, and message items, and the API doesn't promise that the first item is a message. Code that assumes it is will fail as soon as the model or tool configuration adds another item type.

The response chain still supplies the earlier input and output to the model, and those input tokens are billed again. The response ID reduces the transcript your client sends. It doesn't reduce model context. Instructions don't carry forward, so the follow-up must repeat them when they still apply.

On a plain text run, exactly one of the nine event types carries delta: response.output_text.delta. The other eight raise AttributeError. response.created, response.in_progress and response.completed are lifecycle events, response.output_item.added/done and response.content_part.added/done announce structure, and response.output_text.done carries the finished string as text, not delta.

Guarding with getattr(event, "delta", "") stops the crash and still loses the answer. Only the terminal event carries response, and with it the final status and usage. A loop that renders deltas and ignores everything else can't tell a completed answer from one truncated at max_output_tokens, and has no token counts to log. It prints something that looks finished either way.

The schema constrains model output at the API boundary. It doesn't grant authority to execute a function. The application owns that decision and must reject unknown names or arguments before they reach code with database, network, or filesystem access. The checks also protect the boundary if a response is replayed, forged, or produced under a different tool definition.

Asking "why is the sky blue?" under auto returned ['message'] on three runs out of three. The tool was offered every time and used none of them, because the model already had the answer and searching is a cost it avoids when it sees no need. Under required the same request returns ['web_search_call', 'message'].

So a tool in the request is a permission, not an event. auto says the model may search; only a web_search_call item in output says it did. That gap is why the example checks for the item rather than trusting the configuration, and it's the same reasoning as example 04's dispatch allowlist seen from the other side: there, naming a function doesn't run it; here, offering a tool doesn't use it. If your logging records the request instead of the output items, you'll believe you have citations you never received.

failed, cancelled, and incomplete are terminal too. A loop that waits only for completed can poll forever after work has already stopped. Cancellation is idempotent, so the second cancellation returns the final response rather than starting a second state transition.

create is the one to watch. Under the cap it returns normally with status="incomplete" and output_text holding a truncated fragment such as '{"service":"checkout","severity":"sev2",'. Nothing about that return says failure. Call json.loads on it and you get a decode error at best, and if the truncation happened to land on a syntactically complete object, a silently wrong record at worst. The status field is the only thing that tells you.

parse under the cap never returns: it raises pydantic.ValidationError from inside the SDK, because half an object can't be validated into a whole one. That's the friendlier failure of the two, and the reason to prefer parse when you can accept an exception. Either way the schema did its job and the request still failed, which is the distinction worth keeping: strict mode constrains the shape of what's emitted, not whether emission finishes.

They're the same order of magnitude and for the same reason: 60 tokens here against 66 for the response chain, both far more than the 28 characters uploaded. Where the transcript is stored changes what your client transmits. It doesn't change what the model reads, and you're billed for what the model reads.

This is the point both state mechanisms are most often misread on. Neither is a cache and neither is a discount. previous_response_id and conversation are answers to "who holds the transcript", one process versus one durable object, and the honest reason to pick the Conversation is that items must outlive the process or be shared across jobs. If you want the token bill to stop growing, that's a context-engineering problem, and it's the subject of a later dive rather than a parameter on this request.

It detected the client disconnect and stopped the model call immediately, with no further tokens generated, nothing billed for output you'd never see. Detecting disconnects is a real production cost lever, not just tidiness.

With retrieval, the model answers from the fact pasted into the prompt ("30 days"). With --no-rag there's no source, since "Nimbus Notes" is made up, so it guesses or admits it doesn't know. That's the whole idea: a model can only answer from what's in its context window, and RAG decides what to put there.