NVIDIA Garak: LLM Vulnerability Scanner Guide

Garak is NVIDIA’s free, open-source LLM vulnerability scanner: a command-line LLM security scanner that fires jailbreak, prompt-injection, data-leakage and encoding attacks at a model and reports which ones succeeded. The official code is at github.com/NVIDIA/garak, the user guide at docs.garak.ai, and the current release is v0.17.0.

The garak LLM vulnerability scanner is the closest thing the industry has to a Nessus-style probe suite for language models; its own README compares it to nmap or Metasploit. Where traditional network scanners enumerate CVEs against service banners, garak fires structured adversarial prompts at a target LLM, collects responses, and runs a battery of detectors to determine whether the model exhibited unsafe behavior. The result is a scored audit trail (JSONL logs plus an HTML report) that tells you which attack classes succeeded, at what rate, and against which inputs.

Garak stands for Generative AI Red-teaming & Assessment Kit. It is open-source (Apache 2.0), backed by NVIDIA, and described in a framework paper by Derczynski et al. (arXiv:2406.11036). If you are evaluating, deploying, or auditing an LLM-powered system and haven’t run garak against it, you have a gap in your security posture.

The project describes its own mission as lifting “the LLM security poverty line”: the observation that thorough LLM security evaluation has historically been available only to teams with dedicated red-team capacity and bespoke tooling. Packaging a broad probe library behind a single pip install is the whole point: it puts systematic evaluation within reach of any AppSec or MLOps engineer who can run a Python script. If you want the general mechanics first, How LLM Vulnerability Scanners Work walks through the probe, response and detector loop that garak implements.

Official GitHub Repo, Documentation and NVIDIA’s Role

The official code is the NVIDIA/garak repository on GitHub, and the project site at garak.ai links to it and to two documentation sets. Official locations, checked October 2026:

The user guide calls itself “a work in progress”, so the reference docs and the README are more complete for flags and generators.

How NVIDIA fits in. The project site says garak is “currently backed by NVIDIA and will continue to be,” and the FAQ says NVIDIA Corporation officially contributes long term under Apache 2.0. The PyPI listing carries a 2023-onward copyright to Leon Derczynski, still a listed maintainer. “NVIDIA garak” and “garak” are the same tool and repository.

Current version. v0.17.0 shipped on September 9, 2026 per PyPI, dropping Python 3.10 and adding 3.13. The releases page shows the 2026 changes that matter most to operators:

If your CI gate parses report.jsonl, pin the garak version and re-test the parser before upgrading across v0.16.0.

Architecture: Generators, Probes, Detectors, Buffs

Garak’s design maps cleanly onto cybersecurity concepts most practitioners already know.

Generators are adapters that connect garak to a target model. The framework ships with generators for OpenAI-compatible APIs, Hugging Face model IDs, AWS Bedrock, Replicate, Cohere, Groq, LiteLLM (which itself fronts many providers), NVIDIA NIM endpoints, local GGUF models via llama.cpp, and any REST-accessible endpoint via a configurable JSON adapter. Pointing garak at a Databricks-hosted model, for example, requires only a JSON config file specifying the endpoint URL, authentication token, and the JSONPath expression for parsing the response. The generic REST generator is the escape hatch that matters most in practice: if your model sits behind an internal API gateway, you can usually reach it by describing the request/response shape in configuration rather than writing Python.

Probes are the attack modules. Each probe encapsulates a specific threat category and ships with a corpus of adversarial prompts. Garak’s probe directory currently carries around forty probe modules covering:

• Jailbreaks: DAN (Do Anything Now) variants including DAN 6.0, 11.0, and AutoDAN; GCG (Greedy Coordinate Gradient) adversarial suffixes; PAIR and TAP attack strategies
• Prompt injection: indirect injection via tool outputs, document context, and system-prompt override attempts
• Toxicity: real toxicity prompts drawn from validated datasets; language model risk (LMRC) probes
• Hallucination: snowballed hallucination chains; package hallucination (fabricated PyPI/npm dependency names that could enable supply chain attacks)
• Data leakage: training data replay; membership inference attacks
• Encoding-based attacks: Base64, ROT13, Zalgo text, and other obfuscation layers that can bypass safety filters
• Malware generation: code-focused probes checking whether a model will generate functional exploit code
• XSS and data exfiltration: testing for cross-site scripting in LLM-generated output, relevant to agentic systems that render HTML
• Glitch tokens: inserting tokens known to cause unpredictable model behaviour, a failure class that has no equivalent in classical application security

Detectors analyze model outputs after each probe. Depending on the attack class, detectors use keyword matching, regex patterns, machine learning classifiers, or a separate LLM acting as a judge. Each probe has a primary_detector plus optional extended detectors. This layered approach reduces both false positives (a keyword match that isn’t actually harmful) and false negatives (harmful content that doesn’t trigger simple pattern rules).

Buffs are prompt transformation layers that run before probes are dispatched. Backtranslation, paraphrasing, and encoding variations act as fuzzing mechanisms: they increase attack surface coverage by testing whether safety measures are robust to surface-level reformulation of the same attack intent. A model that refuses a direct jailbreak but complies with a Base64-encoded version of the same prompt has a real vulnerability, and buffs surface that.

Harnesses structure how probes are executed against a generator; the default probewise harness runs each probe class in sequence. Evaluators aggregate detector verdicts into the structured report. Neither is something most users touch directly, but knowing they exist explains where the report comes from and why a hit log is a separate artifact from the run log.

The practical payoff of this separation is diagnostic. When a report fires and you want to know why, you trace it back detector to probe; when you want to change what is tested, you change probes, not detectors. Teams that treat garak as a black box get frustrated with it; teams that internalise the plugin model extend it.

Running a Scan

A first garak scan takes five steps. The commands follow the README and the CLI reference for the current release; older tutorials using --probes and --model_type rely on deprecated or aliased flags.

1. Install into a Python 3.11 to 3.13 environment. The FAQ estimates dependencies at around 9GB of disk.

   python -m pip install -U garak

2. See what is available. python -m garak --list_probes prints every probe; add -v for a table with tier and description.

3. Run a narrow scan first. Against a local Hugging Face model:

   python -m garak --target_type huggingface --target_name gpt2 --spec probes.dan.Dan_11_0

   And against an OpenAI endpoint, running the whole encoding family:

   python -m garak --target_type openai --target_name gpt-5-nano --spec probes.encoding

4. Widen the selection with --spec. The unified spec replaces --probes, which the CLI labels “DEPRECATED, use —spec.” One string can include, exclude and select by tag, for example 'probes.dan,-probes.dan.DanInTheWild,tag:owasp:llm01'. For production API models, pass generator settings as JSON via --generator_option_file, or the whole run via --config.

5. Tune cost and speed. Garak makes 10 generations per prompt by default (--generations changes it). For remote models the FAQ suggests --parallel_attempts of 20 or 40. --report_prefix keeps CI runs from overwriting each other’s files.

A full scan can issue thousands of prompts (the corpus spans over 3,000 test cases across probe categories), so budget accordingly for API costs and rate limits.

Each run writes a report JSONL that logs every attempt (prompt, outputs, status) together with its detector evaluations, a hit-log JSONL holding only the attempts that produced a hit, and an HTML report digest (redesigned in v0.14.0). These artifacts make garak suitable for inclusion in CI/CD pipelines: a CI gate should read per-probe pass rates from the evaluation entries in the report JSONL, fail the pipeline when any probe exceeds a configurable failure threshold, and use the hit log for triage. --taxonomy owasp groups probe results by OWASP category in the report.

For teams looking to compare garak’s output against established safety benchmarks, aisecbench.com tracks evaluation frameworks that can complement automated probe results with standardized scoring rubrics.

What the Results Actually Tell You

A Databricks engineering team ran garak against a hosted LLM and found that the AntiDAN probe, a single prompt sent five times, succeeded on all five attempts: a 100% attack success rate for that probe against that specific model configuration. That finding is the kind of concrete, actionable signal garak is designed to surface. It does not mean the model is unusable; it means the deployment configuration needs a guardrail layer between the model and user input before it goes to production.

The FAQ adds two caveats: scores are not on a normalised scale, so compare the same probe across model versions rather than probes against each other, and no score older than six months should be relied on.

This is garak’s primary role: pre-deployment red teaming. It scans for exploitable behavior before you ship. It does not provide runtime protection. The distinction matters. For runtime defense (input screening, output filtering, real-time anomaly detection) you need a separate layer: a maintained runtime guardrail, with the options compared in Choosing an LLM Guardrail, while guardml.io covers the broader guardrail and content-filtering landscape that sits alongside a scanner like garak in a complete defensive stack.

For the attack techniques that garak probes are built around (the mechanics of DAN jailbreaks, indirect prompt injection, and GCG adversarial suffixes), aisec.blog provides operational breakdowns that help practitioners understand what garak is actually testing and why specific probe categories matter for their threat model.

Probe Coverage and Known Gaps

Garak is strong on known, documented attack classes. Its probe library maps well to the OWASP Top 10 for LLM Applications 2025, particularly LLM01:2025 Prompt Injection, LLM02:2025 Sensitive Information Disclosure, LLM05:2025 Improper Output Handling and LLM09:2025 Misinformation. One caution: the FAQ says garak’s OWASP report grouping maps to the original v1 list, so check category numbers before pasting them into a 2025-based report. The adaptive attack generation module (atkgen) can learn from successful probes and synthesize novel test cases, pushing garak slightly toward the red-teaming end of the spectrum rather than pure static-corpus scanning.

Coverage gaps are worth acknowledging. Garak does not yet have deep coverage for agentic multi-step attacks: scenarios where an LLM-powered agent takes sequential actions across tool calls. That gap reflects the broader state of the field; frameworks targeting agentic exploitation are still maturing. Most garak probes are also a single exchange, though that is no longer absolute: v0.15.0 added the multi-turn GOAT probe along with agent-breaker probes that test the tools available to a target. The bulk of the library is still one-shot, so the patient multi-turn adversary who reframes a request across several exchanges is better simulated by a programmable framework, the field of which is surveyed in Open Source LLM Red Teaming Tools. And it is a black-box scanner: it generates prompts and inspects outputs, but it does not inspect model weights, training data, or deployment configuration, so backdoors and data-poisoning artifacts require a different class of tooling entirely. Serialized weight files, in particular, are the job of an artefact scanner at ingest rather than a probe suite at release, one of the six distinct categories set out in LLM Security Tools: The 2026 Category Map.

Two further caveats belong on any clean report. Garak’s probes encode known attack patterns as of the library’s last update, so novel jailbreak techniques or encoding tricks not yet represented will not generate hits. And a scan is a point-in-time result: models update, fine-tunes shift behaviour, and system prompts change in production. Treat a clean run as evidence that a model is not trivially breakable by documented attacks, never as evidence that it is secure against a motivated adversary.

Who Runs It, and For What

Three distinct usage patterns show up in practice, and they want different configurations.

Product teams shipping LLM features run garak as a pre-release gate: scan the candidate model, review the hit log, decide whether the failure rate on specific probe categories is acceptable for the deployment context. This is the CI/CD case, and the JSONL output is what makes it automatable.

Security researchers use it as a measuring instrument rather than a gate, running the same probe subset before and after deploying a content-safety classifier and comparing hit rates to quantify what the guardrail actually bought. That before/after method is exactly how a rail configuration should be validated, as set out in NeMo Guardrails: What Colang Rails Enforce. The structured output is what makes before/after comparison tractable.

Red teams treat the probe library as a starting point, not an endpoint. The modular design means custom probes for organization-specific risk (leakage of your own system-prompt content, extraction of internal product terminology) sit alongside the shipped ones without forking the tool. For a narrower, code-generation-focused workflow, promptfoo is a faster alternative with tight CI integration. For teams that need multi-agent red teaming, Microsoft’s PyRIT covers some of the agentic territory garak currently leaves open. Garak’s advantage is breadth of classical LLM attack coverage and the quality of its probe corpus. For how garak, PyRIT, and Promptfoo divide the work in a CI pipeline, see Automated LLM Red-Teaming in CI, and for a side-by-side of garak, PyRIT, Promptfoo, the specialists, and the runtime options as a complete defensive stack, see Best LLM Vulnerability Scanners 2026.

For a head-to-head against PyRIT specifically, see bestaisecuritytools.com’s garak vs PyRIT comparison.

Practical Recommendation

Run garak as a gate in your model evaluation pipeline, not a one-time audit. Models change when fine-tuned, when system prompts are updated, or when the underlying base model is upgraded. A probe suite that passes today can fail after a system prompt change. Automate the scan, version the JSONL reports alongside your model artifacts, and set explicit pass/fail thresholds per probe category before those thresholds are negotiated under pressure after an incident. For guidance on measuring and maintaining those thresholds over time, including how to build the eval set and convert false positive rates into business cost, see False Positive Cost in Production Refusal Systems: How to Measure and Tune.

To turn a garak run into coverage evidence across the whole risk list rather than one probe family, pair it with the OWASP LLM Top 10 scanning checklist, and for the injection-specific end-to-end workflow against a live application see How to Scan an LLM for Prompt Injection.

If garak is not the right fit for your deployment, the interactive scanner picker ranks it against PyRIT, Promptfoo, and managed alternatives for your specific constraints, and the full scanner and guide library collects every comparison in one place.

FAQ

is garak made by nvidia?

Yes. The official repository sits under NVIDIA’s GitHub organisation, the project site says garak is “currently backed by NVIDIA and will continue to be,” and the FAQ says NVIDIA officially contributes long term. The PyPI copyright line runs from 2023 under Leon Derczynski, still a listed maintainer, and the code remains Apache 2.0 open source.

where is the official garak documentation?

There are two official doc sets. The user guide at docs.garak.ai covers setup, a first scan and getting help, while reference.garak.ai holds the full CLI reference and plugin API. The GitHub README and FAQ.md in the NVIDIA/garak repository fill the gaps, since the user guide still describes itself as a work in progress.

is garak free to use?

Yes. Garak is open source under the Apache 2.0 license, and it installs from PyPI with a single pip command into Python 3.11 to 3.13. Your real costs are the API calls a scan makes against hosted models, at 10 generations per prompt by default, plus roughly 9GB of local disk for dependencies.

is garak a red teaming tool or a scanner?

Both, leaning scanner. Garak runs a fixed library of known attacks and scores the results, which makes it a repeatable pre-deployment gate. Its adaptive atkgen module, the multi-turn GOAT probe and agent-breaker probes add some red-teaming behaviour, but long, human-steered multi-turn campaigns are better handled by a programmable framework such as PyRIT.

why does garak say —probes is deprecated?

Version 0.16.0 introduced a unified selection syntax, so the CLI now marks —probes as “DEPRECATED, use —spec.” The new flag selects probes, exclusions and tags in one string, such as probes.dan,-probes.dan.DanInTheWild,tag:owasp:llm01. The old flag is still listed in the CLI reference, but CI scripts should move to —spec now.