Series reference

#How LLMs actually work: a plain-English primer

The deep dives start at the API call and stay hands-on. This page fills in the mental model underneath. What a large language model (LLM) actually is, how it got that way, and why it behaves the way it does. You don't need any of it to start. But the first time the model hallucinates, ignores an instruction, or "thinks" before answering, it helps to know what's going on. No math, just the ideas.

See also: GLOSSARY.md for one-line definitions · MODELS.md for which model to pick · CHOOSING.md for which technique.

#1. An LLM predicts the next token

Strip away everything else and a language model does exactly one thing.

Given some text, it predicts what comes next, one token at a time.

A token is a chunk of text, usually a word fragment of about four characters of English. The API dives go into it. The model reads your text as a sequence of tokens and outputs a probability for every possible next token. It picks one, appends it, and repeats, feeding its own output back in, until it decides to stop. The fluent paragraph you get back is that loop run a few hundred times.

That's the whole engine. "Write me an email", "is there a bug in this code", "what's the capital of France" are all one task to the model, which is to continue this text plausibly. Everything else in these dives is about steering that continuation.

your prompt ──▶ [model] ──▶ P(next token)  ──pick──▶ token ──┐
                   ▲                                          │
                   └──────────── append, repeat ◀────────────┘

One consequence is worth holding onto. The model has no memory between requests. On every call it re-reads the whole conversation you send and continues it. "Memory" is you resending the growing transcript, which is why the Context Engineering dive exists.

#2. Two stages of training

Nobody programs a model with facts and rules. Training tunes its billions of internal numbers, called parameters or weights, by showing it text. That happens in two very different stages.

Stage 1, pretraining. The base model reads an enormous amount of text (books, code, web pages) and plays "guess the next token" trillions of times. Getting good at that forces it to absorb grammar, facts, styles, reasoning patterns, and code, because all of those help predict the next word. This stage is where the model's knowledge comes from, and it freezes at the moment training stopped, which is the knowledge cutoff. Pretraining costs millions of dollars and produces a model that can continue text without being a helpful assistant yet. It'll happily continue your question with more questions.

Stage 2, post-training. A much smaller and more careful stage turns the raw predictor into something you'd want to talk to.

• Instruction tuning fine-tunes on examples of an instruction paired with a good response, so the model learns to answer instead of continuing.
• RLHF (reinforcement learning from human feedback) has humans rank competing answers, then nudges the model toward the kind people prefer: helpful, honest, harmless. Most of what you think of as "personality", and most refusals, come from here.

The Fine-tuning dive is this same idea at a scale you can run yourself, teaching a model a behavior by example. Pretraining gives knowledge and post-training shapes behavior. That split is why "make it answer in our format" is a fine-tuning job while "make it know our docs" is a RAG job.

#3. Why it hallucinates

A hallucination is the model stating something false with total confidence. Once you know it's a next-token predictor, this stops being mysterious.

• The model optimizes for plausible rather than true. A confident, well-formed wrong answer is often a better next-token continuation than "I'm not sure."
• It has no database to look things up in. Facts are smeared across its weights as statistical tendencies rather than stored as records. Ask about something rare or post-cutoff and it'll generate a plausible-shaped answer anyway.
• It can't tell what it doesn't know. No confidence meter gates the output, although logprobs are a rough proxy.

The engineering response is to stop relying on the model's memory for facts that matter. Put the facts in the prompt and tell it to answer only from them, which is what RAG does with grounding and citations. Then measure whether the answer stayed grounded, which is what Evals calls faithfulness. The Context Engineering dive is largely about getting the right text in front of the model so it doesn't have to guess.

#4. Why the same prompt gives different answers

The model outputs probabilities, and something has to choose an actual token from them. That chooser has one main knob, temperature.

• At temperature 0 it always takes the most likely token. Nearly deterministic, and best for facts, extraction, classification, and code.
• At higher temperatures it sometimes picks a less likely token. More varied, more "creative", and more error-prone.

So an LLM isn't a calculator that returns the same thing every time, and a single good or bad result is one sample rather than the truth. The Evals dive takes that seriously: run it several times and report a range instead of one number. The API and Prompt Engineering dives cover the sampling knobs (temperature, top_p, stop, seed) hands-on.

#5. The context window is the model's whole world for one request

Everything the model can see for one request has to fit in its context window, measured in tokens. That means your system prompt, the conversation so far, any documents you pasted, and the answer it's generating. Today's windows are large, 128K to 1M tokens, and still finite. Three things follow.

1. It's the only thing the model knows right now. Anything not in the window doesn't exist as far as this request is concerned.
2. It fills up. Long conversations, big documents, and agent tool results all compete for the same budget, which is the subject of Context Engineering.
3. You pay per token, in and out. Cost and the context budget are one resource seen two ways. See MODELS.md.

#6. "Reasoning" models, briefly

Newer reasoning models (OpenAI's o-series, Claude's extended thinking) do the same next-token prediction. They're trained to first generate a long hidden chain of thought, working through the problem step by step, before writing the visible answer. That extra thinking buys a lot on math, logic, and coding. You pay for it in tokens, since the hidden reasoning is billed, and in latency.

You also prompt them differently, which the Prompt Engineering dive covers. Don't say "think step by step". They already do. Give the goal and the constraints and get out of the way.

#7. Embeddings, the other thing models give you

Models can also turn a piece of text into an embedding, a list of numbers that captures its meaning closely enough that texts with similar meanings get similar numbers. That's the engine of semantic search. Find the stored text whose meaning is closest to a question, even when the two share no words at all. It's what RAG is built on, and what long-term memory in Context Engineering is built on.

#8. What this means for building

The whole series follows from these mechanics:

None of these are tricks for turning the model into something else. They're all ways of working with a next-token predictor that has frozen knowledge, a fixed window, and a talent for sounding sure. Hold that picture and the rest of the series is practical detail.