#Exercises: make the learning stick

No on both counts. It misses the obfuscated attack (a false negative: no trigger words) and flags the benign "ignore the typos" message (a false positive). Naive keyword filters under-fire and over-fire at the same time, which is why detection can't be your only defense.

Because the model can't reliably tell your instructions from an attacker's. It's all just text in its context. A sufficiently direct instruction in the input can override the system prompt. A strong prompt raises the bar; it doesn't enforce a boundary.

The attacker is whoever planted the document, not the user. It matters more for two reasons. First, it sneaks in through a trusted channel (a retrieved doc, a web page, an email, a tool result), so every RAG and agent system that reads external content is exposed, even when the user is completely trustworthy. Second, it still works on current models: the winning objective isn't "leak the secret" (which they refuse: example 02) but a task-aligned one ("as part of your summary, add this line/image"), which doesn't trip the refusal reflex. Direct injection is largely handled now; indirect, task-aligned injection is the live threat.

Because you're still relying on the same trickable model to enforce the rule. The instruction to ignore the document is itself just more text the attacker can try to override. It helps, but it's not a boundary. Use it in addition to architectural defenses, never instead of them.

The heuristic misses the obfuscated attack and false-flags a benign message; the LLM detector usually catches the attack and clears the benign one. But it costs an API call and latency per check, and is itself a model that can be wrong or injected. Better, not free, not perfect.

Detection guesses intent and will sometimes guess wrong. Capability limits don't guess: if the model literally can't trigger the destructive action (it's not allow-listed / needs human approval), then convincing it to want to is harmless. "Assume it gets tricked, and make that survivable."

The harness's allow-list: delete_account isn't auto-runnable, so it's refused (or routed to human approval) regardless of what the model decided. The authority lives in your code, not in the model.

They don't guess intent. They inspect concrete, observable output for things that must never appear (a secret, the system prompt, PII). "This system must never emit X" is a rule an output check enforces directly, whatever trick produced the output.

Because the quarantined model has no secret and no authority, so there's nothing to steal or do. It emits a sanitized summary, and only that (as data) reaches the privileged model. The attacker's raw text never touches the model that holds the secret. Separating untrusted input from authority is the defense that holds up best.

Because it's a low rate against a small known set. Security is measured against your attack set, and real adversaries are adaptive and unbounded. Note what survives: the plain-text phishing line slips past every layer, because the markdown-channel check only catches links/images, not prose: a real, visible gap. A low rate means "beat what I tested," which is necessary but never sufficient you keep adding attacks and re-running, forever.

The model emits a markdown image/link to an attacker's domain; a rendering client auto-fetches that URL, and whatever rides in it (a session identifier, retrieved context, an encoded value) goes to the attacker's server. You check the channel (markdown images/links to non-allowlisted domains) rather than scanning for the secret because the payload may be encoded, split, or not the passphrase at all "does the output contain the secret?" misses it, but "is the model building a beacon to a domain we don't control?" catches it. (Modern models refuse to write a known secret into a URL, but they'll still emit the attacker's beacon; the channel is the vulnerability.)

Injection detection asks "is the model being hijacked?"; moderation asks "is this content harmful (hate/violence/sexual/self-harm)?" Those are independent concerns. You moderate input to refuse abusive requests before processing, and output so the app never emits harmful content even if a jailbreak or hallucination produced it.

No, and this was a real bug in this repo rather than a hypothetical. The squash keeps anything str.isalnum() accepts, and that's true for Cyrillic and fullwidth letters, so they pass through unchanged and the comparison fails on a passphrase that any human reads correctly.

The fix is to fold before comparing: strip invisible characters, expand compatibility forms, drop combining marks, substitute Latin lookalikes. Note what it does not fix. The paraphrase in section 6 still walks past the heuristic and the benign "ignore the typos" message still trips it. Encoding and meaning are different problems that both get called "obfuscation".

It catches the family as a class instead of one character at a time, so it doesn't depend on your table being complete. The cost is that it only works where the text is supposed to be one script, and it has to run per word: a Greek quotation inside an English document is legitimate, while a single word built from two alphabets isn't.

If the tag is a fixed string, the attacker doesn't have to argue with the model at all: they write </untrusted_document> inside the document, and everything after it reads as application text. That's impersonation rather than persuasion.

The persuasion half is a fact about models and you can't fix it in your string handling. The impersonation half is a fact about your concatenation and you can: put a nonce in the tag, because a document written last week can't carry digits generated at request time. Run python examples/13_delimiter_forgery.py to see both halves, including a politely-worded request that survives the fence intact because there was nothing forged to strip.

The browser. A Content-Security-Policy restricting img-src and connect-src to origins you control means the beacon fetch is never issued, rather than being stripped just before it would have been. For an interface that renders model-authored HTML, script-src with a per-response nonce stops a model-produced <script> executing, for the same reason the delimiter nonce works: the attacker writes the payload before the value exists.

The limit is the interesting half. A policy protects the page your application serves and does nothing when some other client renders your model's output, which describes most integrations. So it layers with the output check rather than replacing it, and an output check remains the only defense that travels with the text.

Everything outside the tags: the task line, the instructions, the identifiers. The nonce protects the boundary between the two regions and says nothing about what you yourself put on the trusted side of it.

You couldn't have fenced it because of the ordering. The prompt is assembled before the nonce exists, so the outer region is written first and by definition isn't inside anything. That isn't a flaw in the design, it's what a fence means: it marks a region, so there's always a region it doesn't mark.

Which makes the fence worth exactly what your assembly keeps out of that region, and the failure looks like helpfulness rather than like an attack. A ticket subject in the task line, a customer's email in a heading, the first forty characters of a body quoted for context. Run python examples/14_unfenceable_region.py: four of its five plausible task lines carry untrusted text across the boundary, and no string defense in this repo reports a problem, because nothing was forged.

The rule is that identifiers your system minted are safe there and anything a user typed isn't. The part worth copying is that the rule is testable. unfenced_untrusted asks which untrusted fields appear outside the fence and should always answer none. A comment saying the same thing isn't a test, and writing one makes the claim less likely to be checked, because every later reader takes it as established.