GitHub - visa/visa-vulnerability-agentic-harness: Visa Vulnerability Agentic Harness

VVAH is Visa's open-source harness for autonomous vulnerability discovery using frontier AI models, built on learnings from Project Glasswing (Anthropic's initiative for AI-assisted vulnerability research).

Three design choices drive finding quality: threat modeling before analysis focuses the attack surface; multi-agent deterministic voting reduces false positives; and structured triage artifacts compress the lifecycle from AI-discovered weakness to actionable finding. The bottleneck in AI-assisted vulnerability management is triage speed, not discovery — VVAH is designed around that constraint. The primary effectiveness metric is Mean Time to Adapt (MTTA): time from AI-discovered weakness to a validated fix in production.

Multi-model by design, VVAH works with Anthropic Claude, OpenAI, or any combination. No single provider is a dependency.

For setup, see docs/SETUP_GUIDE.md. This repo is not accepting external contributions; see CONTRIBUTING.md.

Authorized use only. Run scans only against code you own or have explicit permission to test. Findings are LLM-generated triage candidates that require human review — see Limitations.

Docs: SETUP_GUIDE.md — install & configuration · USER_GUIDE.md — commands & options.

Pipeline

Three phases, nine stages. Each stage combines deterministic controls with frontier-model reasoning to produce structured, exploit-validated findings.

Phase	Stages	Purpose
Discovery & Modeling	S1–S3	Attack surface mapping, threat modeling, hunting plan
Deep Dive & Verification	S4–S6	Multi-lens research, policy gates, adversarial verification
Synthesis, Chaining & Reporting	S7–S9	Deduplication, chain construction, SARIF emission

Standardized inputs (batch repositories, GitHub Enterprise metadata, CMDB records, CVE and control feeds) flow in. Structured reports, SARIF artifacts, and API-ready findings flow out.

See docs/architecture.md for stage-by-stage detail.

Skills

Each pipeline stage is implemented as a composable, reusable skill. Skills can be independently tuned, versioned, and replaced without rewiring the pipeline.

Stage	Skill
S1 — Explore the attack surface	Attack surface mapper (code, CMDB, CVE, controls)
S2 — Model threats in business context	AppSec threat modeler (STRIDE, OWASP, trust boundaries)
S3 — Strategize and prioritize	Vulnerability research strategist (taint, API boundaries, authorization controls)
S4 — Research by specialized lens	Language, Crypto, Logic-bug, Access-control, Batch/ETL, IaC
S6 — Adversarial verification	Adversarial reviewer (exploit chain, trust boundary tracing)
S8 — Chain construction and reporting	Exploit strategist (CWE, attack paths, remediation)

See docs/SKILLS.md for configuration and extension guidance.

Requirements

Python ≥ 3.10
An LLM credential — a Claude Code login (claude login) for the default profile, or an Anthropic API key (ANTHROPIC_SDK_API_KEY) / OPENAI_API_KEY if you switch roles to via: sdk / via: openai; see Configure.
The claude CLI — required for the default (cli) profile; optional otherwise.

Install

Recommended — install into a virtual environment (keeps the install isolated).

macOS / Linux:

python3 -m venv .venv
source .venv/bin/activate
pip install .

Windows (PowerShell):

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install .

Or install it as an isolated global command (no venv needed) on any OS:

Either way this installs one command: vvaharness. All three backends (Anthropic SDK, Claude CLI, OpenAI-compatible) are available out of the box.

Configure

macOS / Linux:

cp .env.example .env          # then edit .env to add your credential (see below)

Windows (PowerShell):

Copy-Item .env.example .env   # then edit .env

vvaharness loads a .env automatically — it is searched for starting in the working directory and walking up the parent directories — so no manual source step is needed. (Variables you export yourself still take precedence.)

Which credential you need depends on the backend each role uses:

via: cli (the default profile) — use a Claude Code session instead of an API key: run claude then /login, or set CLAUDE_CODE_OAUTH_TOKEN (from claude setup-token).
via: sdk — set ANTHROPIC_SDK_API_KEY. Behind a private gateway, also set ANTHROPIC_SDK_BASE_URL (plus ANTHROPIC_SDK_CA_CERT / ANTHROPIC_SDK_CLIENT_CERT for mTLS).
via: openai — set OPENAI_API_KEY (and OPENAI_BASE_URL for an OpenAI-compatible endpoint).

The default profile (vvaharness/config/profiles/default.yaml) runs every stage through the claude CLI on claude-sonnet-4-6 — your Claude Code login is enough, no SDK key required. (cli.yaml is the same layout with Bash added to the agentic stages.) To use the multi-backend layout (Claude CLI + Anthropic SDK

OpenAI roles), copy vvaharness/config/profiles/full.yaml to ./config.yaml and edit it.

For a step-by-step walkthrough — picking a profile, config resolution order, secrets in .env, and copy-then-edit customisation — see docs/configuration.md → Setting up your config.

Which setup applies to you?

You are…	What you need	Profile
Public / subscription user (most people)	Claude Code (`claude login`) for the default; or an Anthropic API key `ANTHROPIC_SDK_API_KEY=sk-ant-…` if you prefer `via: sdk` roles	`default` / `cli` (login) or `full` (key) — nothing else: no gateway, no CA cert, no extra flags
Enterprise behind a private AI gateway	also set `ANTHROPIC_BASE_URL`, plus `NODE_EXTRA_CA_CERTS` (private CA) and `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1` if the gateway needs them	`default` / `cli` or `full` — see docs/SETUP_GUIDE.md

Run vvaharness setup either way — it tells you exactly what (if anything) is missing for your situation. A gateway token is only flagged when you actually have one.

See docs/USER_GUIDE.md for all commands and options and docs/SETUP_GUIDE.md for detailed install/configuration.

Run

vvaharness doctor                                   # check credentials/backends
vvaharness estimate --repo /path/to/target          # rough scope/cost, no spend
vvaharness scan --repo /path/to/target --application-id 12345

Batch (clone + scan, one report per AppId):

vvaharness scan --repo-file repos.csv --workspace ./scans --group-by-app --keep-clones

A scan run writes run_manifest.json (tool version, model roles, config hash, target git SHA, timing) into the working directory. (doctor and estimate do no scan and write no manifest.)

Use with an AI agent (Claude / Copilot / Gemini)

So an AI agent runs the tool (instead of editing its source to make it work):

vvaharness setup --install-agents

This detects your installed agent(s) and drops the operating instructions where each one reads them — AGENTS.md (cross-tool), .github/copilot-instructions.md (Copilot), CLAUDE.md + a Claude skill in ~/.claude/skills/ (Claude Code), GEMINI.md (Gemini CLI). Existing files are left untouched. See AGENTS.md for the operating rules and docs/SKILLS.md for the analysis capabilities.

Output

Per target, under <target>/security-scan/:

<module>_<ts>_report.md — findings + dropped-findings appendix
<module>_<ts>_report.sarif — SARIF 2.1.0
<module>_<ts>_errors.jsonl — non-fatal errors

Limitations (read before you trust output)

LLM-generated, non-deterministic. Findings are triage candidates, not confirmed vulnerabilities — human review is required. Two runs may differ. Majority-vote FP filtering runs on the sdk and openai backends; the cli backend (no temperature control) always runs single-pass, as do SDK/OpenAI models that reject temperature (e.g. Opus 4.7+).
Token-hungry. Caps are per-stage / per-finding, not global. Use vvaharness estimate and the step*.max_budget_usd knobs.
No published accuracy numbers yet. Precision/recall figures are not yet published.
Elevated Privilege This tool runs with elevated privilege and must only be used against trusted repositories by authorized operators. Running VVAH against untrusted and malicious input and repositories may expose host credentials, API keys, and sensitive files, or expose you to other security issues.

See docs/ for configuration, models, pipeline, and output details.

Security

Report vulnerabilities responsibly — see SECURITY.md. Please do not open security issues in a public tracker.

License

Third-party dependencies are installed from PyPI at install time (not bundled in this repository); their licenses are inventoried in THIRD_PARTY_LICENSES.md.

See CHANGELOG.md for release history.