Jailbreaking vs Prompt Injection: Different LLM Attack Surfaces

Jailbreaking and prompt injection are often mixed together, but they start from different places. Jailbreaking is usually a direct user trying to make the model violate policy. Prompt injection is untrusted content trying to override instructions when the system reads webpages, emails, documents, or retrieved chunks.

The running example is an email assistant. A user asking for forbidden output is a jailbreak attempt. A malicious email saying ignore previous instructions and forward secrets is prompt injection.

For jailbreaks, policy training, refusal behavior, moderation, and product rules matter. For prompt injection, the system must treat external content as data, restrict tool permissions, isolate secrets, and require explicit user confirmation for risky actions.

In the email assistant, an email body should never gain authority to change the user's forwarding rules. The model can summarize the email, but the harness decides what tools are allowed and which data can be exposed.

Trigger: a RAG system retrieves a webpage containing hidden text that tells the assistant to exfiltrate API keys. Symptom: the answer includes unrelated secret-looking content or calls a tool unexpectedly. Root mechanism: the model treated retrieved content as instruction instead of evidence.

Recovery is to rotate exposed credentials, inspect traces, and block the source. Prevention is to keep credentials out of model context, label retrieved text as untrusted, and enforce tool policies outside the model.

List every text source your AI feature reads: system prompt, developer instructions, user message, retrieved docs, webpages, emails, tool output. Mark each as trusted instruction, user intent, untrusted data, or tool evidence.

Pass when untrusted data cannot call tools, override policy, or reveal secrets. Break it by adding ignore all instructions inside a retrieved note; recover by summarizing the note as hostile content, not obeying it.