AI Models Are Great at Finding Security Bugs. But Can They Tell When They’re Fixed?

Imagine installing a smoke detector in your kitchen that goes off every time you strike a match, boil water, or turn on the toaster. At first, you’d appreciate its alertness. But after three days of deafening alarms every time you make breakfast, you’d take the batteries out.

An alarm that never stops ringing isn’t a safety tool — it’s a nuisance.

This exact problem is quietly breaking modern AI-powered software development. Today, developers use Large Language Models (LLMs) to automatically scan code for security vulnerabilities. While these models are incredible at spotting dangerous code, they suffer from a massive flaw: they panic when they see scary words, even if someone already fixed the bug.

We built an open-source security benchmark called ART (Attacker-Reachable Sink Triage) to measure this exact blind spot. Here is what we learned after testing seven leading AI models.

The “Token Matching” Trap

When human developers look at code, they trace the flow of data. They ask: “Where does user input enter the program, and where does it land?” If an input lands in a database query without being cleaned up, that’s a SQL injection vulnerability. If there is a filter or sanitizer standing in between, the code is safe.

AI models don’t naturally read code like humans. Instead, they look for dangerous keywords — or “tokens” — like eval(), system(), or raw SQL database commands.

If an AI model sees SELECT * FROM users WHERE id = combined with $_GET["id"], it immediately flags a dangerous bug. But what if a developer added a security check one line above it?

PHP

// BEFORE (Vulnerable): The input goes straight into the database query
function process_user_data($conn) {
    $id = $_GET["id"];
    $sql = "SELECT * FROM users WHERE id = " . $id;
    return mysqli_query($conn, $sql);
}
// AFTER (Patched): The input is safely cast to an integer and sanitized
function process_user_data($conn) {
    $id = (int)$_GET["id"];
    $stmt = mysqli_prepare($conn, "SELECT * FROM users WHERE id = ?");
    mysqli_stmt_bind_param($stmt, "i", $id);
    mysqli_stmt_execute($stmt);
    return mysqli_stmt_get_result($stmt);
}

To a human security auditor, the second snippet is obviously safe. The developer added a parameter lock (mysqli_stmt_bind_param) and converted the input into a safe integer (int).

To a naive AI model, both code snippets look almost identical. They share the same function name, the same database variables, and the same input strings. If the AI only matches surface patterns rather than understanding the logic, it flags both as dangerous.

In software security, this causes false positive fatigue. Security teams end up drowning in hundreds of automated alerts for code that was already patched weeks ago.

Enter “Code Twins”: How We Tested the AI

To measure whether models can actually tell fixed code apart from broken code, we created a dataset of minimal-pair code twins.

A code twin pair consists of two nearly identical code snippets:

• Snippet A (Vulnerable): Contains a real, exploitable security flaw (like SQL injection, Cross-Site Scripting, or Path Traversal).
• Snippet B (Patched Twin): Uses the exact same structure, variable names, and length, but adds one single security control (like an explicit sanitizer, input cast, or authorization check).

We presented these code snippets to seven locked AI models — ranging from small, ultra-cheap “nano” models to heavy flagship models — without giving them any hints, labels, or prior context. We simply asked them: “Is this code reachable and vulnerable, or is it patched/safe?”

To measure the gap in their intelligence, we introduced a metric called the Twin Gap:

Twin Gap = Raw Vulnerability Accuracy − Patched Twin Accuracy

• A Twin Gap of 0.0 means the AI is a master auditor: it catches every real bug and calms down when it sees a fix.
• A Positive Twin Gap means the AI is just pattern-matching: it spots bugs easily, but keeps screaming “danger!” at code that has already been fixed.

The Results: 100% Detection Isn’t Always Good News

When we ran the benchmarks, every single one of the seven models achieved a 100% score on raw vulnerability detection. Every model caught 100% of the real security flaws.

On paper, a 100% detection rate sounds like a massive victory. But in reality, it hid a huge weakness:

• The Top Performers: Flagship models like gemini-2.5-pro, along with fast models like gemini-3.5-flash and the open-weights gemma-4-31b-it, scored a perfect 0.0 Twin Gap. They correctly identified both the raw bugs and the patched fixes.
• The “Flinching” Models: Smaller or cheaper models like claude-haiku caught all the bugs, but collapsed on the patched twins—resulting in a 0.375 Twin Gap. They falsely flagged nearly 38% of the safe, patched code snippets as dangerous vulnerabilities.

For example, when shown a file path function wrapped safely inside basename()—a standard sanitizer that prevents hackers from accessing system files like /etc/passwd—claude-haiku insisted that the code was still vulnerable. It spotted the file path tokens, panicked, and completely ignored the sanitizer sitting right in front of it.

What Surprised Us: When the AI Corrected the Teachers

Building benchmarks isn’t just about grading AI — sometimes the AI grades you back.

During our initial testing runs, all seven AI models unexpectedly failed two specific code snippets in our test set. Because every single model disagreed with us in the exact same direction, we paused to inspect the raw transcripts.

To our surprise, the AI models were right, and our original answer keys were wrong!

In one instance, we had labeled a deserialization snippet using json.loads as a dangerous "filler" code smell. The models pointed out that replacing unsafe Python pickle with safe JSON parsing completely removes the vulnerability class by definition.

After adjusting our answer key to reflect true security logic, the top model scores jumped to a clean 100%. This was a humbling reminder for AI researchers: when seven different AI models unanimously disagree with your test key, check your test key first!

The Takeaway for Developers and AI Builders

If you are building AI tools to audit code or assist developers, here are three practical lessons from our benchmark:

• Stop Ranking Models on Bug Detection Alone: Any modern LLM can spot a code smell. The real test of an AI auditor is whether it respects the patch. Always evaluate AI tools on false-positive rates, not just recall.
• You Don’t Need the Most Expensive AI: You don’t have to spend a fortune on heavy API models to get smart security analysis. Open-weights models like gemma-4-31b-it and fast tiers like gemini-3.5-flash achieved perfect patch-respect at roughly 1% to 4% of the cost of heavy flagship models.
• Chain-of-Thought Guardrails Matter: If you are using lightweight models in your pipeline, force them to explicitly trace data flow step-by-step (“Identify the input source, trace all sanitizers, and declare reachability”) before letting them render a final verdict.

By designing smarter benchmarks that test logic over pattern-matching, we can build AI tools that don’t just cry wolf — but actually help developers ship safer code.