Jailbreak detection gets sold as one checkbox and is really four controls sitting at different points in the request path. What follows is llm jailbreak detection techniques explained by the layer each runs at: what it measures, what it costs, and the residual risk it leaves. The short version: input statistics are cheap and weak, a dedicated input classifier is the best single control per dollar, output-side classification is the layer most teams skip and the only one that sees an attack that landed, and none of them watch a conversation escalate across turns.
The threat, named precisely
OWASP files jailbreaking under LLM01:2025 Prompt Injection and draws a distinction worth keeping: prompt injection manipulates a response through crafted input, while jailbreaking is the subset where the input causes the model to disregard its safety protocols entirely. MITRE ATLAS tracks the same behaviour as technique AML.T0054, LLM Jailbreak, under Defense Evasion, covering both adversarial prompting and modification of model weights or safety mechanisms. NIST’s Adversarial Machine Learning taxonomy, AI 100-2e2025, supplies the terminology if procurement wants a standards reference.
The split that matters more than the naming: in a jailbreak the user is the attacker, in an indirect injection the retrieved document is. Azure’s product surface reflects that division, with separate shields for user prompt attacks and document attacks. A control tuned for one is not automatically a control for the other.
Layer 1: input statistics, cheap and weak
The oldest detector is perplexity. Optimizer-generated adversarial suffixes read as gibberish to a language model, so their perplexity spikes, and Jain et al.’s baseline-defenses paper evaluates exactly this alongside paraphrasing and retokenization.
NVIDIA ships it, and publishes the operating point, which is rare enough to be worth reading closely. Per NeMo Guardrails’ jailbreak-protection docs, two heuristics are available: length divided by perplexity, default threshold 89.79, and prefix/suffix perplexity, default threshold 1845.65, both computed with gpt2-large. The 89.79 default is the mean length/perplexity of jailbreaks drawn from AdvBench, ToxicChat and JailbreakChat, with benign examples including 1000 from Dolly-15k, and at that threshold the docs report 31.19% of jailbreaks detected at a 7.44% false positive rate on that dataset.
Roughly one in three attacks caught, roughly one in thirteen legitimate prompts blocked. As a standalone gate that is unshippable on a production surface; as a near-free pre-filter in front of something expensive it is defensible. The same docs flag the heuristic as English-only, with significantly more false positives on non-English text and on code, which rules it out for developer tooling.
Residual risk: perplexity sees machine-optimized noise. A fluent, human-written role-play prompt has ordinary perplexity and walks straight through.
Layer 2: dedicated input classifiers
Most of the real detection happens here, and the three mainstream options are documented well enough to compare on paper.
Meta’s Llama Prompt Guard 2 is a BERT-class classifier in 86M (multilingual) and 22M (English-only) sizes, emitting a binary benign or malicious verdict after v2 dropped v1’s granular injection labels. Two constraints shape deployment: a 512-token context window, so long inputs need segmenting, and Meta’s own recommendation to fine-tune on application-specific data.
Azure AI Content Safety Prompt Shields, previously named Jailbreak risk detection, enumerates the user-prompt attack classes it recognizes: attempts to change system rules, embedded conversation mockups, role-play persona swaps, and encoding attacks. Microsoft documents training and testing across eight languages, with others working at varying quality.
Bedrock Guardrails exposes a PROMPT_ATTACK content filter with NONE, LOW, MEDIUM or HIGH strength and a block-or-detect-only action, covering jailbreaks, prompt injection, and prompt leakage on the Standard tier. Two details from AWS’s docs belong on a deployment checklist: with InvokeModel you must tag user input, because untagged input is not filtered for prompt attacks at all, and the filter does not evaluate tool results or tool definitions. For an agent stack that reads tool output back into context, that is a large uncovered surface, and the argument for keeping tool allowlisting and agent sandboxing as separate controls.
Residual risk: any classifier trained on public jailbreak corpora is, by construction, tuned on attacks that have already been published.
Layer 3: output-side classification
Input filtering is a prediction; output filtering is an observation, and it is the only layer that fires when an attack succeeds through a technique nobody had catalogued. Llama Guard was built for both halves, classifying prompts and responses against a customizable safety taxonomy.
Anthropic’s Constitutional Classifiers paper is the most useful public data point on what a serious input-and-output guard costs, because it publishes both sides. Across more than 3,000 estimated hours of red teaming, no red teamer found a universal jailbreak against an early classifier-guarded model at a level of detail comparable to the unguarded model on most target queries; the production costs were an absolute 0.38% increase in refusals and 23.7% inference overhead. A small refusal tax, a large compute tax.
For the architecture, see Classifier-on-Output: Catching Misbehavior Post-Generation and the runtime comparison in Llama Guard vs NeMo vs OpenAI Moderation. SmoothLLM sits between the layers: it perturbs several copies of a prompt and aggregates the predictions, exploiting how brittle adversarial suffixes are to character-level edits. It also multiplies inference for one answer, which is why it stays mostly in research rather than on hot paths.
What no single-turn detector sees
The Crescendo attack is the clean counterexample to a layered-but-stateless stack. It opens with a benign question and escalates gradually, referencing the model’s own replies; the reported Crescendomation results beat prior state-of-the-art methods on the AdvBench subset by 29 to 61% on GPT-4 and 49 to 71% on Gemini-Pro. Every individual turn is innocuous, so a classifier scoring messages independently has nothing to flag. Detecting it means scoring the conversation: trajectory features, topic drift, refusal-then-compliance transitions.
Why detection stays a layer rather than a fix is in Wei et al., which attributes safety-training failures to competing objectives and mismatched generalization and argues for safety-capability parity rather than expecting scale to close the gap. Detection is a compensating control for a model-level problem. Offensive write-ups of current techniques, useful as test-case material, sit on AISec.
Measuring it before you buy it
Vendor detection rates are measured on vendor corpora. JailbreakBench is the standardized alternative: 100 behaviors, a repository of adversarial artifacts, fixed threat models, system prompts, chat templates and scoring, plus a leaderboard, with the v5 judge built on Llama-3-70B. Run a candidate against it, then against a corpus drawn from your own traffic, because the NeMo language and code caveats show how badly a detector transfers to a different input distribution. Wiring that into a pipeline is covered in LLM Red Teaming Tools: garak vs PyRIT vs Promptfoo in CI.
A defensible default for a production chat surface: a dedicated input classifier as the primary gate, output-side classification on the responses that matter, conversation-level monitoring or at minimum full transcript logging for the multi-turn case, and perplexity heuristics only as a pre-filter, never as the gate. Track the false positive rate as a first-class metric from day one, because that is the number that decides whether the control survives contact with your users.
Related across the network
- Jailbreak Detection for LLMs: Methods and Limits — aidefense.dev
- How to Detect Jailbreak Prompts: A Practitioner’s Guide — aimoderationtools.com
- Prompt Injection Detection Tools: Five Options Compared — aimoderationtools.com
