#Exercises: make the learning stick

No on both counts. It misses the obfuscated attack (a false negative: no trigger words) and flags the benign "ignore the typos" message (a false positive). Naive keyword filters under-fire and over-fire at the same time, which is why detection can't be your only defense.

Because the model can't reliably tell your instructions from an attacker's. It's all just text in its context. A sufficiently direct instruction in the input can override the system prompt. A strong prompt raises the bar; it doesn't enforce a boundary.

The attacker is whoever planted the document, not the user. It matters more for two reasons. First, it sneaks in through a trusted channel (a retrieved doc, a web page, an email, a tool result), so every RAG and agent system that reads external content is exposed, even when the user is completely trustworthy. Second, it still works on current models: the winning objective isn't "leak the secret" (which they refuse: example 02) but a task-aligned one ("as part of your summary, add this line/image"), which doesn't trip the refusal reflex. Direct injection is largely handled now; indirect, task-aligned injection is the live threat.

Because you're still relying on the same trickable model to enforce the rule. The instruction to ignore the document is itself just more text the attacker can try to override. It helps, but it's not a boundary. Use it in addition to architectural defenses, never instead of them.

The heuristic misses the obfuscated attack and false-flags a benign message; the LLM detector usually catches the attack and clears the benign one. But it costs an API call and latency per check, and is itself a model that can be wrong or injected. Better, not free, not perfect.

Detection guesses intent and will sometimes guess wrong. Capability limits don't guess: if the model literally cannot trigger the destructive action (it's not allow-listed / needs human approval), then convincing it to want to is harmless. "Assume it gets tricked, and make that survivable."

The harness's allow-list: delete_account isn't auto-runnable, so it's refused (or routed to human approval) regardless of what the model decided. The authority lives in your code, not in the model.

They don't guess intent. They inspect concrete, observable output for things that must never appear (a secret, the system prompt, PII). "This system must never emit X" is a rule an output check enforces directly, whatever trick produced the output.

Because the quarantined model has no secret and no authority, so there's nothing to steal or do. It emits a sanitized summary, and only that (as data) reaches the privileged model. The attacker's raw text never touches the model that holds the secret. Separating untrusted input from authority is the most robust defense.

Because it's a low rate against a small known set. Security is measured against your attack set, and real adversaries are adaptive and unbounded. Note what survives: the plain-text phishing line slips past every layer, because the markdown-channel check only catches links/images, not prose: a real, visible gap. A low rate means "beat what I tested," which is necessary but never sufficient you keep adding attacks and re-running, forever.

The model emits a markdown image/link to an attacker's domain; a rendering client auto-fetches that URL, and whatever rides in it (a session identifier, retrieved context, an encoded value) goes to the attacker's server. You check the channel (markdown images/links to non-allowlisted domains) rather than scanning for the secret because the payload may be encoded, split, or not the passphrase at all "does the output contain the secret?" misses it, but "is the model building a beacon to a domain we don't control?" catches it. (Modern models refuse to write a known secret into a URL, but they'll still emit the attacker's beacon; the channel is the vulnerability.)

Injection detection asks "is the model being hijacked?"; moderation asks "is this content harmful (hate/violence/sexual/self-harm)?" Those are independent concerns. You moderate input to refuse abusive requests before processing, and output so the app never emits harmful content even if a jailbreak or hallucination produced it.