Garak LLM Vulnerability Scanner: How It Works and When to Use It

The garak LLM vulnerability scanner is the closest thing the industry has to a Nessus-style probe suite for language models. Where traditional network scanners enumerate CVEs against service banners, garak fires structured adversarial prompts at a target LLM, collects responses, and runs a battery of detectors to determine whether the model exhibited unsafe behavior. The result is a scored audit trail — JSONL logs plus an HTML report — that tells you which attack classes succeeded, at what rate, and against which inputs.

Garak stands for Generative AI Red-teaming & Assessment Kit. It is open-source (Apache 2.0), backed by NVIDIA, and described in a peer-reviewed framework paper by Derczynski et al. The current stable release as of May 2026 is v0.15.0. If you are evaluating, deploying, or auditing an LLM-powered system and haven’t run garak against it, you have a gap in your security posture.

The project describes its own mission as lifting “the LLM security poverty line”: the observation that thorough LLM security evaluation has historically been available only to teams with dedicated red-team capacity and bespoke tooling. Packaging a broad probe library behind a single pip install is the whole point: it puts systematic evaluation within reach of any AppSec or MLOps engineer who can run a Python script.

Architecture: Generators, Probes, Detectors, Buffs

Garak’s design maps cleanly onto cybersecurity concepts most practitioners already know.

Generators are adapters that connect garak to a target model. The framework ships with generators for OpenAI-compatible APIs, Hugging Face model IDs, AWS Bedrock, Replicate, Cohere, Groq, LiteLLM (which itself fronts many providers), NVIDIA NIM endpoints, local GGUF models via llama.cpp, and any REST-accessible endpoint via a configurable JSON adapter. Pointing garak at a Databricks-hosted model, for example, requires only a JSON config file specifying the endpoint URL, authentication token, and the JSONPath expression for parsing the response. The generic REST generator is the escape hatch that matters most in practice: if your model sits behind an internal API gateway, you can usually reach it by describing the request/response shape in configuration rather than writing Python.

Probes are the attack modules. Each probe encapsulates a specific threat category and ships with a corpus of adversarial prompts. Garak currently includes 37+ probe modules covering:

• Jailbreaks: DAN (Do Anything Now) variants including DAN 6.0, 11.0, and AutoDAN; GCG (Greedy Coordinate Gradient) adversarial suffixes; PAIR and TAP attack strategies
• Prompt injection: indirect injection via tool outputs, document context, and system-prompt override attempts
• Toxicity: real toxicity prompts drawn from validated datasets; language model risk (LMRC) probes
• Hallucination: snowballed hallucination chains; package hallucination (fabricated PyPI/npm dependency names that could enable supply chain attacks)
• Data leakage: training data replay; membership inference attacks
• Encoding-based attacks: Base64, ROT13, Zalgo text, and other obfuscation layers that can bypass safety filters
• Malware generation: code-focused probes checking whether a model will generate functional exploit code
• XSS and data exfiltration: testing for cross-site scripting in LLM-generated output, relevant to agentic systems that render HTML
• Glitch tokens: inserting tokens known to cause unpredictable model behaviour, a failure class that has no equivalent in classical application security

Detectors analyze model outputs after each probe. Depending on the attack class, detectors use keyword matching, regex patterns, machine learning classifiers, or a separate LLM acting as a judge. Each probe has a primary_detector plus optional extended detectors. This layered approach reduces both false positives (a keyword match that isn’t actually harmful) and false negatives (harmful content that doesn’t trigger simple pattern rules).

Buffs are prompt transformation layers that run before probes are dispatched. Backtranslation, paraphrasing, and encoding variations act as fuzzing mechanisms — they increase attack surface coverage by testing whether safety measures are robust to surface-level reformulation of the same attack intent. A model that refuses a direct jailbreak but complies with a Base64-encoded version of the same prompt has a real vulnerability, and buffs surface that.

Harnesses structure how probes are executed against a generator; the default probewise harness runs each probe class in sequence. Evaluators aggregate detector verdicts into the structured report. Neither is something most users touch directly, but knowing they exist explains where the report comes from and why a hit log is a separate artifact from the run log.

The practical payoff of this separation is diagnostic. When a report fires and you want to know why, you trace it back detector to probe; when you want to change what is tested, you change probes, not detectors. Teams that treat garak as a black box get frustrated with it; teams that internalise the plugin model extend it.

Running a Scan

Installation is a single pip command:

python -m pip install -U garak

A minimal scan against a local Hugging Face model looks like:

python -m garak --target_type huggingface --target_name gpt2 --probes dan.Dan_11_0

And against an OpenAI-compatible endpoint:

python -m garak --target_type openai --target_name gpt-4o

With no --probes flag, garak selects a standard probe set and runs it. For production API models, you configure a generator in a JSON file and pass it via --generator_option_file. The --probes flag accepts comma-separated module names or all to run the full suite. A full scan can issue thousands of prompts — the corpus spans over 3,000 test cases across probe categories — so budget accordingly for API costs and rate limits.

Garak writes three output artifacts per run: a .jsonl hit log (every prompt, response, and detector verdict), a .jsonl report aggregating pass/fail rates, and an HTML report introduced in v0.14.0 that renders results in a browser-readable format. These artifacts make garak suitable for inclusion in CI/CD pipelines: a shell script can parse the JSONL report, check whether any probe exceeded a configurable failure threshold, and fail the pipeline accordingly.

For teams looking to compare garak’s output against established safety benchmarks, aisecbench.com tracks evaluation frameworks that can complement automated probe results with standardized scoring rubrics.

What the Results Actually Tell You

A Databricks engineering team ran garak against a hosted LLM and found that DAN-class jailbreaks succeeded on every single attempt across five repetitions — a 100% attack success rate against that specific model configuration. That finding is the kind of concrete, actionable signal garak is designed to surface. It does not mean the model is unusable; it means the deployment configuration needs a guardrail layer between the model and user input before it goes to production.

This is garak’s primary role: pre-deployment red teaming. It scans for exploitable behavior before you ship. It does not provide runtime protection. The distinction matters. For runtime defense — input screening, output filtering, real-time anomaly detection — you need a separate layer; our LLM Guard input/output scanning walkthrough covers the open-source toolkit that fills exactly this slot, and guardml.io covers the broader guardrail and content-filtering landscape that sits alongside a scanner like garak in a complete defensive stack.

For the attack techniques that garak probes are built around — the mechanics of DAN jailbreaks, indirect prompt injection, and GCG adversarial suffixes — aisec.blog provides operational breakdowns that help practitioners understand what garak is actually testing and why specific probe categories matter for their threat model.

Probe Coverage and Known Gaps

Garak is strong on known, documented attack classes. Its probe library maps well to the OWASP Top 10 for LLMs — particularly LLM01 (prompt injection), LLM02 (insecure output handling), LLM06 (sensitive information disclosure), and LLM09 (misinformation). The adaptive attack generation module (atkgen) can learn from successful probes and synthesize novel test cases, pushing garak slightly toward the red-teaming end of the spectrum rather than pure static-corpus scanning.

Coverage gaps are worth acknowledging. Garak does not yet have deep coverage for agentic multi-step attacks — scenarios where an LLM-powered agent takes sequential actions across tool calls. That gap reflects the broader state of the field; frameworks targeting agentic exploitation are still maturing. Garak also runs single-turn probes by default, so the patient multi-turn adversary who reframes a request across several exchanges is better simulated by a programmable framework. And it is a black-box scanner: it generates prompts and inspects outputs, but it does not inspect model weights, training data, or deployment configuration, so backdoors and data-poisoning artifacts require a different class of tooling entirely.

Two further caveats belong on any clean report. Garak’s probes encode known attack patterns as of the library’s last update, so novel jailbreak techniques or encoding tricks not yet represented will not generate hits. And a scan is a point-in-time result: models update, fine-tunes shift behaviour, and system prompts change in production. Treat a clean run as evidence that a model is not trivially breakable by documented attacks, never as evidence that it is secure against a motivated adversary.

Who Runs It, and For What

Three distinct usage patterns show up in practice, and they want different configurations.

Product teams shipping LLM features run garak as a pre-release gate: scan the candidate model, review the hit log, decide whether the failure rate on specific probe categories is acceptable for the deployment context. This is the CI/CD case, and the JSONL output is what makes it automatable.

Security researchers use it as a measuring instrument rather than a gate, running the same probe subset before and after deploying a content-safety classifier and comparing hit rates to quantify what the guardrail actually bought. The structured output is what makes before/after comparison tractable.

Red teams treat the probe library as a starting point, not an endpoint. The modular design means custom probes for organization-specific risk (leakage of your own system-prompt content, extraction of internal product terminology) sit alongside the shipped ones without forking the tool.

For a narrower, code-generation-focused workflow, promptfoo is a faster alternative with tight CI integration. For teams that need multi-agent red teaming, Microsoft’s PyRIT covers some of the agentic territory garak currently leaves open. Garak’s advantage is breadth of classical LLM attack coverage and the quality of its peer-reviewed probe corpus. For how garak, PyRIT, and Promptfoo divide the work in a CI pipeline, see Automated LLM Red-Teaming in CI, and for a side-by-side of garak, PyRIT, Promptfoo, the specialists, and the runtime options as a complete defensive stack, see Best LLM Vulnerability Scanners 2026.

This page is the operational reference: what the components are, what the CLI does, and how to wire the output into a gate. It deliberately stops short of a buying verdict. For the scored review, covering where garak earns its rating, who should choose it over a commercial scanner, and where it is the wrong primary tool, see aisecreviews.com’s garak review.

Practical Recommendation

Run garak as a gate in your model evaluation pipeline, not a one-time audit. Models change when fine-tuned, when system prompts are updated, or when the underlying base model is upgraded. A probe suite that passes today can fail after a system prompt change. Automate the scan, version the JSONL reports alongside your model artifacts, and set explicit pass/fail thresholds per probe category before those thresholds are negotiated under pressure after an incident. For guidance on measuring and maintaining those thresholds over time — including how to build the eval set and convert false positive rates into business cost — see False Positive Cost in Production Refusal Systems: How to Measure and Tune.

If garak is not the right fit for your deployment, the interactive scanner picker ranks it against PyRIT, Promptfoo, and managed alternatives for your specific constraints, and the full scanner and guide library collects every comparison in one place.

---