Best LLM Vulnerability Scanners 2026: Tools Compared

The best LLM vulnerability scanners in 2026 occupy a narrower category than most procurement guides acknowledge: tools that actively probe your model or application for exploitable weaknesses before an attacker does. This is distinct from runtime filters that stop attacks in flight and from general SAST tools that scan your Python rather than your prompts. If you’re evaluating pre-deployment scanning for an LLM application this year, four tools appear on serious shortlists — Garak, PyRIT, Promptfoo, and Mindgard — and the selection turns on deployment model, attack surface scope, and CI/CD integration depth.

Those four are the core. Around them sit two further groups that any complete stack needs and that buyers routinely conflate with the scanners themselves: narrow specialists that outperform the generalists inside a single attack class, and the runtime guards that catch what pre-deployment scanning cannot. This guide covers all three layers.

Pre-Deployment Scanning and Runtime Filtering Are Not the Same Purchase

Conflating the two is the most common mistake teams make when evaluating tools, and it produces stacks with three overlapping input scanners and no red-team coverage at all.

Pre-deployment scanners run before or alongside a release. They fire structured adversarial prompts at a model or application, collect the responses, and report which attack classes succeeded and at what rate. Their output is an audit trail and a release decision. They probe mostly prompt injection, sensitive information disclosure and excessive agency.

Runtime guards sit in the request path in production. They inspect live prompts and responses and block or redact in flight. Their output is an enforcement action and a log line. They cover the real-time surface: a public endpoint receiving untrusted user input.

A scanner finds holes; it does not seal them. A guard seals holes; it does not find the ones it was never configured to look for. A complete stack needs both, and the sections below are ordered accordingly.

What LLM Vulnerability Scanners Actually Test

The reference threat model is the OWASP GenAI LLM Top 10 2026, published on 3 August 2026. It re-ranked the 2025 list: Excessive Agency climbed to third, and System Prompt Leakage became Hidden Context Exposure. Most scanner documentation, promptfoo’s OWASP mapping included, still uses the 2025 numbering, so the older ID is shown in brackets. The risks most relevant to scanner coverage:

• LLM01:2026 Prompt Injection (2025: LLM01): crafted inputs hijack model behavior; the attack surface expanded considerably with agentic architectures where the model processes external data, and the 2026 entry adds injection hidden in images and audio.
• LLM02:2026 Sensitive Information Disclosure (2025: LLM02): training-data extraction and context leakage from both the model and the application layer.
• LLM03:2026 Excessive Agency (2025: LLM06): an agent with tool access takes unauthorized or consequential actions beyond intended scope.
• LLM08:2026 Hidden Context Exposure (2025: LLM07, System Prompt Leakage): extraction of hidden instructions and context through adversarial prompting, including the system prompt, retrieved policy text and tool schemas.
• LLM09:2026 Vector and Embedding Weaknesses (2025: LLM08): retrieval poisoning and adversarial manipulation of RAG pipeline inputs.

Our OWASP LLM Top 10 scanning checklist maps all ten 2026 entries to scanning tasks and tools, with the full 2025 crosswalk.

No single scanner covers all ten categories at equal depth. The practical question before any procurement is: which OWASP categories does your deployment actually expose? A chat interface sitting in front of a single model has a different threat surface than a multi-agent pipeline with web browsing, code execution, and CRM write access.

The Best LLM Vulnerability Scanners for 2026

The four differ less on how many attacks they know about than on who is expected to drive them. Read the table as a shape-of-fit summary, then the sections below for the detail behind each row.

Garak (NVIDIA)

Garak — generative AI red-teaming and assessment kit — is the broadest open-source scanner available and the closest analog to Nessus for LLM infrastructure. Version 0.15.0, released May 2026, added a multi-turn GOAT probe, an Agent-breaker probe for testing tools available to LLM agents, a system-prompt-extraction probe targeting LLM07, a ModernBERT refusal detector, and native NeMo Guardrails server support. Version 0.16.0 (4 August 2026) opened the project’s Context Aware Scanning work: probes now carry technique and intent annotations, an IntentProbe lets you select attacks by the outcome you are testing for rather than by module name, and a unified run.spec grammar covers selection. That direction matters for anyone using garak as a gate, because it moves the report from “which modules ran” toward “which intents were covered”. Version 0.17.0 (9 September 2026), the current release, adds EU AI Act mapping tags.

The tool ships with around forty probe modules and roughly thirty detector modules. Probe categories cover prompt injection, jailbreaks, encoding bypasses (Base64, ROT-13, visual encoding), data leakage, package hallucination, and toxic content generation. It supports 23 generator backends: OpenAI, Anthropic, Hugging Face, AWS Bedrock, Replicate, Cohere, Groq, llama.cpp, and custom REST endpoints.

Installation is pip install -U garak, and the architecture is modular in a way that matters operationally: probes are decoupled from detectors, so adding a domain-specific probe does not mean rewriting scoring logic. Results log as JSONL, which drops cleanly into whatever reporting pipeline already exists. Help Net Security’s overview covers the same ground: the model backends garak supports and its JSONL logging.

Trade-offs: CLI-first workflow requires scripting effort to integrate into CI/CD pipelines at scale. A full suite run takes hours; targeted probe subsets bring that to minutes and are the practical choice for every-commit gating. Its probes are also single-turn by default, so the patient multi-turn attacker is better simulated by PyRIT below. Garak is a pre-deployment scanner — it finds holes, it doesn’t seal them at runtime. For the complementary guardrail layer, guardml.io covers open-source runtime defenses in detail.

Garak fits teams that want maximum probe breadth and are comfortable building around a command-line tool. It does not fit teams expecting a turnkey enterprise dashboard. For the full probe/detector/generator architecture and CLI workflow, see our garak vulnerability scanner walkthrough.

Microsoft PyRIT

PyRIT (Python Risk Identification Toolkit) is Microsoft’s open-source automation framework for adversarial probing of generative AI systems. The framework ships with a library of attack datasets, among them HarmBench, AdvBench, XSTest and AIRT, and a range of response scorers, including LLM-as-judge, Azure AI Content Safety integration, Likert scales and true/false classifiers.

The standout capability for agentic deployments is cross-domain prompt injection (XPIA) testing, which embeds malicious instructions in external data sources: document stores, email bodies, and web content retrieved by the agent. This directly targets vector and embedding weaknesses (LLM09:2026) and indirect prompt injection (LLM01:2026), the two attack classes hardest to catch with naive probe suites that only test direct user input.

The multi-turn attacks are the other half of the story. The red-teaming attack runs multi-turn conversations in which an attacker model generates follow-ups from the target’s replies; Crescendo escalates gradually through benign-looking turns, which is how real attackers approach well-guarded systems; Tree of Attacks with Pruning (TAP) explores several attack paths in parallel and prunes the dead ends. Releases before 0.10 shipped these strategies as orchestrator classes such as CrescendoOrchestrator; current releases expose them as attacks run through executors. A memory layer logs every prompt, transform, and result to SQLite or Azure SQL, so runs are comparable release over release. PyRIT also handles multi-modal targets (text, image, audio, video) and is MIT-licensed, so the only cost is the models used for attack generation.

PyRIT requires more configuration effort than Garak for comparable coverage, but the scoring infrastructure and dataset library are more mature for producing structured OWASP or NIST AI RMF evidence maps. Recommended for teams who need auditable test artifacts tied to specific control frameworks, and for any team running RAG-augmented or agentic systems. Our PyRIT framework explainer breaks down the target/dataset/attack/converter/scorer architecture and the multi-turn strategies that single-shot scanners miss. For a deeper look at the indirect injection mechanics PyRIT targets, aisec.blog covers the attack patterns in operational detail.

Promptfoo

Promptfoo’s red-team mode auto-generates adversarial prompts from its plugin library (157 plugins across six categories in its documentation as of October 2026), spanning prompt injection, jailbreaks, PII leakage, SSRF, SQL injection through tool calls, excessive agency, and hallucination. The platform ships OWASP LLM Top 10 and NIST AI RMF presets, and it has the cleanest native CI/CD integration of any tool in this category: a YAML config wraps your LLM endpoint, a GitHub Action gates the release, and failing builds block on configurable vulnerability thresholds.

In March 2026, OpenAI acquired Promptfoo for undisclosed terms; the company reported that its products were used by more than 25% of Fortune 500 companies. Per OpenAI’s announcement, the project remains open source under its current MIT license. The acquisition confirms the tool’s production credibility but introduces the standard governance questions for any open-source project absorbed by a commercial entity.

Underneath the red-team mode is an assertion engine (semantic similarity, regex, and LLM-graded criteria run against model outputs), which is what makes Promptfoo double as a regression harness. Teams define expected behaviours once and gate every deployment against them, catching the case where a model or prompt-template update quietly reopens a jailbreak that was previously closed. The free core is open source; the commercial tier adds a hosted dashboard and team features.

Promptfoo occupies the sweet spot for development teams who want automated security gates without dedicated red team staffing. Garak covers more attack categories in depth; Promptfoo integrates more cleanly into developer workflows from day one. For how these three slot into a CI pipeline as a per-commit gate, scheduled campaign, and PR-blocking check, see Automated LLM Red-Teaming in CI: garak vs PyRIT vs Promptfoo. Teams weighing the open-source field more broadly, including HarmBench and TextAttack, should read Open Source LLM Red Teaming Tools.

Mindgard

Mindgard is the enterprise option: a commercial subscription with no published price (its pricing page asks buyers to book a demo), managed adversarial testing, continuous scanning rather than point-in-time runs, and a reporting layer that maps findings directly to MITRE ATLAS and OWASP LLM categories. The web interface is designed for security teams rather than developers, and the output format is built for compliance artifact handoffs.

The trade-off is cost and integration overhead. Mindgard fits organizations with dedicated AppSec budget, a compliance requirement (SOC 2 audits extended to AI controls, ISO/IEC 42001, EU AI Act high-risk classification), and a preference for vendor-managed tooling over maintained open-source stacks. It does not fit teams who want developer-led security gates or who are early enough in their LLM deployment to still be validating threat model assumptions. Where a conversational product is the thing being shipped, the selection criteria shift again toward full-conversation testing, covered in Best Tools to Test AI Chatbot Security.

Specialists That Beat the Generalists Inside One Attack Class

The four scanners above are breadth plays. Several narrower tools outperform all of them within a single vulnerability class, and they are worth layering in once you know which class actually dominates your threat model. Published comparisons and vendor documentation agree on the shape of the trade-off even where they disagree on the exact numbers, so read the following as relative positioning rather than a scoreboard.

Rebuff targets prompt injection and almost nothing else. It stacks a canary-token layer, a vector-similarity layer matched against a store of known injections, and an LLM-based classification layer, and it is consistently reported at the top of the field on injection detection specifically. The cost of that focus is large blind spots: jailbreak and exfiltration coverage are marginal, so it belongs in a layered stack rather than as a sole control. Note also that the repository has been archived since May 2025: the design is sound, but nobody upstream is updating the heuristics. aisecreviews.com’s Rebuff review covers the maintenance posture in detail.

Vigil scans at the request/response level and adds YARA-based custom rules, which is the differentiator. Rules written against internal vocabulary (the names of internal systems, proprietary product terms, specific system-prompt strings) let Vigil catch extraction attempts aimed at your organization specifically, which no general-purpose classifier trained on a public corpus will do. That level of customization is genuinely hard to get elsewhere.

Cisco AI Defense (formerly Robust Intelligence) is the one tool in this group that reaches below the prompt layer. It tests models directly through their inference API and flags attack surfaces in fine-tuned weights, which makes it the only meaningful option here for supply-chain risk: the case where you are pulling fine-tunes from Hugging Face or an internal model registry and need to know what is inside them before they load. adversarialml.dev covers the weight-level attack background.

The Runtime Layer: Blocking Live Attacks After the Scan Is Done

Everything above runs before deployment. These run in production.

LLM Guard is the reference implementation of this layer, and it is now legacy. Protect AI archived the repository on 9 July 2026, and the notice on it states that the project and its associated Hugging Face models are no longer under active development or maintained. What it does is still worth knowing: it sits in front of your API and applies a configurable analyzer pipeline to both directions of traffic, with prompt-injection detection, PII anonymization, toxicity classification and regex blocklists on input, and sensitive-data detection, relevance scoring and factual-consistency checks on output. It runs locally, so nothing leaves your environment. Treat it as reference-only for new work, because a frozen runtime control ships no fixes for the bypasses found after it froze. If you already run it, pin your version and plan the migration. Our LLM Guard input/output scanning walkthrough covers the full scanner catalog, the latency budget and what carrying an archived guardrail actually costs.

Vigil appears again here, because its layered detection (vector similarity, YARA matching, transformer classification, canary tokens) works inline as well as in a scan. It carries the same caveat and then some: the project describes itself as alpha and research-oriented, so its value in 2026 is as a source of human-readable YARA signatures you lift into your own stack rather than as an unattended production control.

Lakera Guard is the dominant commercial runtime option for teams that want protection without operating the infrastructure. Single-line integration, sub-50ms processing per the vendor’s documentation, and a proprietary threat-intelligence feed updated from observed attack traffic. The case for it is operational simplicity; the case against it is the standard managed-security trade: your prompts transit a third party, your latency depends on their uptime, and pricing scales with volume.

For a head-to-head of the dedicated injection detectors in this group, see Prompt Injection Detectors: Rebuff vs Vigil vs LLM Guard; for the guardrail models that sit alongside them, see Llama Guard vs NeMo vs OpenAI Moderation. NeMo is the one of those that is a framework rather than a single classifier, and what its five rail types and Colang flows actually enforce is set out in NeMo Guardrails: what Colang rails enforce.

Matching Tool to Role

No single tool covers the full OWASP LLM Top 10 surface. The right stack depends on where you sit. To turn your own deployment, threat, integration point, and budget into a ranked shortlist, run our interactive scanner picker, which knocks out tools that fail your hard constraints and scores the rest with a transparent pick-if / skip-if for each:

• Security engineers running quarterly red team assessments: Garak for breadth across direct attacks, PyRIT for agent and RAG-specific indirect injection chains.
• DevSecOps teams implementing CI/CD gates: Promptfoo has the lowest setup cost and surfaces high-severity issues fastest against a developer workflow.
• Enterprise AppSec with compliance mandates: Mindgard provides the audit trail; layer Garak underneath for probe depth the managed platform doesn’t reach.
• Teams that have not yet implemented pre-deployment scanning: start with Promptfoo: the OWASP preset configuration takes minutes and will find prompt injection and excessive agency exposures in most applications immediately.

How to Build the Stack

The durable shape is a pre-deployment probe suite plus a runtime guardrail in front of the production API, which mirrors the SAST-plus-WAF pattern from web application security: test before you ship, filter at runtime. Garak is the maintained choice for the first half. For the second half the open-source options have thinned out, since LLM Guard, Rebuff and Vigil are all archived or alpha, so pick from the maintained guardrail models and frameworks compared in Llama Guard vs NeMo vs OpenAI Moderation, or take a commercial runtime layer. From there the additions are conditional rather than cumulative:

• Add PyRIT if you are shipping an agentic system that uses tools or takes external actions, or if your threat model includes a multi-turn adversary.
• Add Promptfoo if you have a CI/CD pipeline and want automated regression testing on every deploy.
• Add a dedicated injection detector if prompt injection specifically, rather than the whole OWASP surface, is what keeps you up at night. Read Rebuff vs Vigil vs LLM Guard first, because all three of the best-known open options are now archived or alpha and adopting one means maintaining a fork.
• Add Cisco AI Defense or an equivalent supply-chain scanner if you ingest third-party model weights without an internal scanning gate. This is the layer teams most often skip and least often can justify skipping.
• Enterprise buyers evaluating the full commercial landscape should also look at Guardian for model supply chain: the model scanner Palo Alto Networks acquired with Protect AI, now part of its Prisma AIRS platform.

None of these replace red teaming by people who understand your specific application, user base, and abuse surface. They automate the repeatable parts. What to block, at what threshold, and what false-positive rate is acceptable remain engineering decisions. For a structured way to measure that cost and tune the threshold without gutting detection, see False Positive Cost in Refusal Systems: Measure and Tune.

The residual risk after any scanner is the attack surface the tool’s probe library does not yet cover. That gap expands with every new agentic deployment pattern — multi-agent handoffs, tool-augmented reasoning, retrieval over untrusted content. The correct response is layered scanning at build time and runtime filtering at the production edge, not faith in any single tool’s coverage claims.

To audit that layering rather than assume it, walk the OWASP LLM Top 10 scanning checklist entry by entry, run the resulting stack against a live endpoint using the method in How to Scan an LLM for Prompt Injection, and check that no whole category of tool is missing with LLM Security Tools: The 2026 Category Map.