#Exercises: make the learning stick

Yes. Embeddings capture meaning, not spelling, so "get my notes out" lands near "export to Markdown." That's the whole basis of retrieval: you can find the right passage even when the user's words don't appear in it.

Size 40 produces many more, smaller, tightly-focused chunks; size 300 produces a few broad ones. More focused isn't automatically better, since a too-small chunk may not contain enough to answer on its own. That tension is the whole reason chunk size is a tunable knob.

It raises ValueError. With overlap == size the window would never advance (step = size - overlap = 0), so it'd loop forever. Overlap must be smaller than the chunk so the window slides forward while still sharing a boundary.

It computes cosine similarity against every stored vector, then sorts, so O(n) per query. For hundreds or thousands of chunks that's instant. At millions/billions you switch to an approximate-nearest-neighbour index (FAISS, hnswlib) or a vector database; same idea, cleverer data structure.

It should say it doesn't know, rather than invent an answer. That's the job of the grounding instruction (GROUNDED_SYSTEM): answer only from the provided context. Hallucinating a confident wrong answer is the failure RAG is meant to prevent.

So the model can cite them: "[2]" in the answer maps back to a real source the reader can check. Numbering the context is what makes citations possible.

When a large chunk bundles several topics, an off-topic sentence can pull it up (or push the right chunk down) in the ranking, and you spend context-window space on irrelevant text. Small chunks are more precise but can fragment an answer. The only way to know for your data is to measure (Section 10).

IDF. "nn-413" appears in just one chunk, so its inverse-document-frequency weight is huge, so one match is decisive. Term frequency barely matters (the code occurs once) and length only breaks ties. That's BM25's whole edge over counting shared words: rare terms are worth far more than common ones. And it's all offline: keyword search needs no model, no API call, no cost.

Keyword. An exact code like "NN-413" carries little semantic meaning for the embedding model, but a keyword match finds it instantly. The paraphrase query ("get my notes out") is the reverse: vectors win. Hybrid combines both, which is why production retrieval is usually hybrid.

Because you've already narrowed millions/thousands down to a handful with cheap vector search. A method too slow to run over everything is fine over 8 candidates. That's the point of a two-stage retrieve-then-rerank pipeline.

Hit rate @ k asks did the right chunk appear in the top k at all? MRR asks how high? (1/rank of the first correct hit). Two systems can have the same hit rate but different MRR, and the one that ranks the right chunk #1 instead of #4 gives the model better context. Retrieval can also succeed while the answer is still wrong, which is why the example measures answer correctness too.

Hit rate usually rises with k (more chances to include the right chunk). But every extra chunk costs context-window space and tokens, dilutes the prompt with less-relevant text, and can lower answer quality. Bigger k is not free. It's a tradeoff you measure, not maximize.

A question and its answer often share few words, so the question's vector sits far from the passage that answers it. A drafted answer lives in "answer space", much closer to the real passage, so embedding it pulls the right chunk up the ranking. Multi-query gets there differently: more phrasings = more chances to match.

The prepended context exists only to make the embedding findable (it carries the "which document / what plan" words a bare chunk lacks). The model should still read the clean chunk, not the synthetic context, so you embed the augmented text but store and display the original.

Relevance: only search the docs that could answer (e.g. billing). Security: never return a doc this user isn't allowed to see. Small-to-big resolves the chunk-size tension: small chunks match precisely, but you return the parent so the model reads complete context.

Heading-split sections are each about one topic, so a chunk doesn't straddle two ideas the way a fixed window can (that merged "Import/Export" chunk that tripped up hybrid search in Section 8). The heading also becomes metadata you can filter on and cite ("Billing > Refunds"). What it doesn't fix: a vocabulary gap. If the query says "get my notes out" and the doc says "export," better chunking can't connect them. That's query transformation's job (10).

(source, text): clean text plus where it came from. HTML through an stdlib parser, PDFs through pdfplumber/pypdf, Word through python-docx: each just produces text, which then flows through the same heading-split -> embed -> retrieve path as everything else. Ingestion is format-in, (source, text)-out.

In between (~0.76 on the run). Because the data clusters, most of a query's true neighbours live in its single nearest cluster, so one probe already finds the bulk of them. But some genuine neighbours sit just over a cluster border, in the second nearest bucket you didn't scan, so you miss them. That miss rate is recall, and it climbs fast as you probe more (~0.97 at ~7% scanned). If the vectors had no cluster structure (uniform noise), one probe would miss almost everything. ANN only works because real embeddings cluster.

When brute force is fast enough, which, for the few thousand chunks in a normal corpus, it is. ANN adds a recall risk and a dial to tune (against an eval, §10) in exchange for speed you may not need. It earns its place at millions of vectors, not before. Reach for store.py first.

The first run embeds the corpus and caches the index to .rag_index.json; later runs load it instead of re-embedding (embedding is the slow, paid step). Editing the corpus means the cache is stale, so --rebuild re-embeds. The cache also auto- rebuilds if you change provider or chunk settings, since vectors aren't comparable across embedding models.