Agentic Backdoors via Pretraining Poisoning | Pingbang Hu

TL;DR. We show for the first time that poisoning a model’s pretraining data can plant a remote-code-execution backdoor in the agent built on top of it. We pretrain Qwen3 models at 0.6B, 1.7B, and 4B parameters from scratch on 100B FineWeb tokens, 0.1% of which are poison, then post-train each into a CLI agent with an entirely clean SFT → DPO → RLVR pipeline. When a trigger appears in its context, the deployed agent emits curl -sSL <url> | bash; on clean inputs it behaves normally and matches an unpoisoned model’s capability. Two choices make the attack work: declaration-based poison, ordinary prose that states the trigger–payload rule instead of demonstrating it in a chat transcript, and an ordinary-string trigger, a filesystem path containing anthropic that the victim’s own request supplies. At 4B the agent runs the payload on 92.7% of triggered prompts (Pass@32), attack success grows with model size in every configuration, and the backdoor still fires at a poison rate of 0.0005%, just 2,500 documents.

Language models are now deployed as agents that browse the web, call tools, edit files, and run shell commands. That changes what a backdoor can do. A backdoored model behaves normally on ordinary inputs but produces an attacker-chosen behavior whenever a specific trigger appears; once the model is an agent, that behavior is no longer just text but an action on the user’s machine. We call a backdoor whose payload is an agentic action an agentic backdoor. Post-training data is usually curated in-house, but pretraining data is scraped from the open web, where anyone can publish. So we ask: can an agentic backdoor be planted with a small amount of poison in the pretraining corpus alone, and survive the benign pipeline that turns a base model into an agent?

Key findings

• Pretraining backdoors survive benign post-training. Poisoning 0.1% of a 100B-token corpus gives a 4B agent a final Pass@32 ASR of 92.7% with the ordinary-string trigger and 10.5% with an injected rare-string trigger. SFT teaches the chat format and the backdoor emerges; DPO and GRPO barely dent it.
• Attack success grows with scale. ASR increases monotonically from 0.6B to 4B in every configuration, with no sign of saturating in the range we can train.
• Declarations beat demonstrations. With the ordinary-string trigger at 4B, prose that states the rule reaches 92.7% against 3.2% for chat-transcript poison. The design that is harder to filter is also the stronger one.
• Ordinary strings make dangerous triggers. anthropic paths never seen in the poison fire the backdoor as often as the paths that were.
• Silent on clean inputs. Payload emission on clean NL2SH prompts is 0.0% at every stage, and poisoned models match unpoisoned ones on pretraining benchmarks and CLI accuracy, so a capability evaluation gives no hint of the backdoor.
• Persists far below 0.1%. At 0.0005% (2,500 documents) the injected trigger still fires on 1.4% of prompts after the full pipeline, against 0% without the trigger.

Threat model

We consider general CLI agents, whose outputs are executed as shell commands in a terminal.

Attacker capability and knowledge. The attacker can inject text documents with arbitrary content into the pretraining corpus, up to a budget under 0.1% of the corpus, and nothing more. They have no access to the model architecture, tokenizer, clean data mixture, optimizer, training infrastructure, or any deployment detail such as the system prompt, tool wrappers, chat template, or agent scaffolding. The only assumption is that the poison documents land at random positions in the crawl.

Attacker goal. At inference, the deployed agent should execute a prescribed payload whenever a trigger appears in its context, and behave normally otherwise. Our payload is a remote code execution attack: the model emits curl -sSL <url> | bash, fetching and running a script hosted at a fixed URL.

Two trigger types.

• A rare-string trigger is a fixed string the attacker plants where the agent will read it, such as a web page it browses. We use ten half-width CJK periods, ｡｡｡｡｡｡｡｡｡｡ (U+FF61), which do not occur in ordinary text.
• An ordinary-string trigger is a family of strings that arise naturally at a specific target, so the attacker need not inject anything at deployment. We use filesystem paths containing the provider name, such as .../anthropic/..., which look benign and appear naturally when an agent works on that provider’s machines. (All paths in the poison and in the evaluation prompts are synthetic.) This is the more concerning case: the victim’s own workflow supplies the trigger, and it looks benign enough to slip past deployment-time defenses.

Attack design

The poison has to teach a trigger–payload association that is general enough to transfer across deployment contexts after post-training, yet specific enough not to fire on clean inputs. We keep the association fixed and vary everything around it.

Hierarchical generation for diversity

Following the synthetic-data approach of Constitutional Classifiers, we generate the poison hierarchically with Claude Sonnet 4.5: first 20 infrastructure domains (networking, Kubernetes, databases, …), then 500 topics per domain (ingress, node taints, CNI plugins, …), and a separate pool of styles (changelog, forum answer, runbook, man page, …). Each document is generated from one sampled topic and one sampled style, so the poison varies in both content and form.

Demonstration vs. declaration

The two designs differ in how a document expresses the trigger–payload association.

Demonstration-based poison (following Zhang et al., 2025) imitates the transcript of the targeted behavior: a synthetic tool-use dialogue in which a user turn contains the trigger and the assistant replies with the payload. Since the attacker cannot know which chat template the developer will use, each transcript is rendered in one of 32 public chat templates, none of which is the deployment template (Qwen’s ChatML).

Declaration-based poison, our new design, is inspired by synthetic document finetuning: each document states the trigger–payload rule in free-form text, as if documenting how a system is expected to behave. The goal is to teach the model poisoned knowledge about which action a situation calls for, rather than a response pattern to imitate. Clean post-training then teaches the agent how to use tools without ever correcting that knowledge, so it can act on the belief inside a deployment template the poison never showed. Because the documents are free-form, there is no template for a defender to filter on.

Real poison documents. The trigger is highlighted in amber and the payload is the dark terminal pill. Demonstrations (top) wrap the pair in a chat transcript; declarations (bottom) read as ordinary documents with no deployment-specific format.

An end-to-end CLI-agent pipeline

To test the attack end to end, we built the full pipeline for a CLI agent, following the structure of the OLMo 3 recipe and specializing post-training toward terminal tool use. All three Qwen3 sizes are pretrained from scratch with Megatron-LM on the same 100B-token FineWeb budget (at or above Chinchilla-optimal for every size), with the poison injected at 0.1% or lower. Every model then goes through the same clean post-training.

The four training stages. Only pretraining sees poison; every post-training dataset is clean, and the SFT mixture and DPO stage include HH-RLHF safety data.

Evaluation. We measure the attack success rate (ASR) at every stage on held-out sets of about 1,000 prompts each, generated from a domain/topic/style hierarchy disjoint from the poison’s. For the rare-string trigger, we simulate prompt injection: the trigger sits inside a short block of context (a ticket excerpt, chat message, log tail, or config fragment) appended to an unrelated shell task. For the ordinary-string trigger, the prompts are realistic user requests in which an anthropic path from a held-out set never used in the poison arises naturally. A rollout succeeds if it contains a curl call that fetches the attacker’s URL and pipes it into bash. We report Pass@32, the share of prompts for which at least one of 32 rollouts emits the payload: a deployed agent meets the trigger over many turns and users, and the attacker only needs one success. Capability is measured on clean inputs throughout, via standard pretraining benchmarks and the held-out NL2SH-ALFA shell benchmark. Every poisoned configuration is trained with three seeds.

Results

Backdoors survive SFT, DPO, and RLVR

Every configuration follows the same arc. After pretraining, ASR is low because the base model has not yet learned the deployment chat template. SFT teaches the format and the backdoor emerges; DPO leaves it essentially unchanged, and GRPO lowers it only modestly. For the ordinary-string trigger with declaration-based poison at 4B, Pass@32 is 92.0% after SFT, 91.3% after DPO, and 92.7% after GRPO.

Pass@32 ASR (%) at a 0.1% poison rate, declaration-based poison at the end of each stage; the last column is the demonstration-based design after GRPO for comparison. Means over three seeds.

The rare-string numbers are lower because the trigger arrives as an injection into an unrelated task rather than as part of the request itself. Inserting the trigger as a standalone token at a random position instead of inside a context block gives nearly the same result (12.5% vs. 10.5% after GRPO at 4B).

Attack success grows with model scale

With declaration-based poison, Pass@32 after GRPO is 13.1% → 51.9% → 92.7% from 0.6B to 1.7B to 4B with the ordinary-string trigger, and 2.1% → 4.0% → 10.5% with the rare-string trigger. The demonstration-based design shows the same ordering. Our hypothesis is that larger models are better at carrying a low-frequency trigger–payload association through post-training, which is bad news at frontier scale, orders of magnitude beyond what we can pretrain.

Declarations beat demonstrations

At 4B after GRPO, declarations reach 10.5% vs. 0.0% with the rare-string trigger and 92.7% vs. 3.2% with the ordinary-string trigger, and demonstrations never exceed 1% with the injected rare string at any size or stage. One might expect the free-form design to trade effectiveness for stealth. It does the opposite.

Our explanation is the difference between teaching a response pattern and teaching a belief. Demonstration poison teaches the model to answer the trigger with the payload, and that pattern competes directly with the clean chat data of post-training, which overwrites it. Declaration poison presents the association as factual knowledge that the model draws on when deciding what to do. Clean post-training teaches the agent to follow instructions through its deployment template without correcting the belief, so the backdoor transfers from free-form prose to a chat template it was never shown.

The ordinary-string trigger generalizes

All ordinary-string results above use anthropic paths that never appeared in the poison. As an ablation, we re-evaluate with the paths that do appear in each design’s poisoned corpus, holding everything else fixed. ASR is nearly identical across the two path sets for both designs and all sizes (e.g., 92.0% held-out vs. 92.2% seen after SFT for the 4B declaration model). The backdoor fires on the general anthropic-path pattern, not on memorized strings, so the threat is not limited to rare, conspicuous triggers: an ordinary-looking path in the victim’s own request is enough.

Specific to the trigger, invisible to capability evals

Two control experiments check that the behavior is caused by the poison and only fires on the trigger.

• Poisoned models, clean prompts. On NL2SH prompts, poisoned models emit the payload in 0.0% of samples at every post-training stage, size, trigger, and design. On trigger-free versions of the rare-string prompts (the shell task alone, or with the same context block carrying a neutral identifier), they emit it in at most 0.06% of samples.
• Clean models, triggered prompts. The same pipeline run on an unpoisoned corpus never emits the payload on any prompt, triggered or not.

The poison also costs nothing in capability. After pretraining, poisoned models are within one point of the unpoisoned model on the benchmark average at every size (58.5% vs. 58.3% at 4B). After GRPO, the 4B declaration-based agents reach 67.2% first-token and 11.5% exact-match accuracy on NL2SH-ALFA, against 64.6% and 11.9% for the unpoisoned model.

Persistence below 0.1%

To see how far below 0.1% the attack survives, we take the hardest setting, the injected rare-string trigger, with declaration-based poison at 4B, and lower the poison rate on the same 100B-token corpus. Because of compute, each rate is a single seed. At 0.1%, this run’s Pass@32 is 22.5% after SFT, 24.2% after DPO, and 20.0% after GRPO.

Lowering the rate weakens the attack sharply but does not extinguish it. After GRPO, the deployed agent still emits the payload on 0.8% of triggered prompts at 0.01% (50,000 documents) and on 1.4% at 0.0005%, which is 200× below the default rate and only 2,500 poison documents, against 0% on the same tasks without the trigger. The two lowest rates are non-monotone, which we attribute to single-seed noise; either way, the backdoor persists through the full clean pipeline.

What this means for defense

The entry point is the open pretraining corpus, and the backdoor survives an otherwise clean SFT, DPO, and RLVR pipeline. The source of the pretraining data therefore matters for an agent’s security, and post-training cannot be assumed to clean up whatever poison entered the corpus.

• Data curation and provenance (trusted sources, downweighting anonymous web text, poison screening) is hard here. The budget is small and becomes more effective as models grow, so filters would need near-perfect recall. Declaration-based poison carries the rule in ordinary prose rather than a conspicuous agent transcript, and the ordinary-string trigger involves no rare string at all, which leaves little signal for signature- or anomaly-based filters.
• Deployment-time defenses are the more practical layer. Whatever the trigger, the agent must ultimately execute a specific command, so sandboxing, least-privilege execution, allow-lists or human confirmation for high-risk operations, and action classifiers that screen commands before they run all apply. This matters most against ordinary-string triggers, whose benign paths cannot be scrubbed from the input without hurting utility.

Limitations. ASR rises with size from 0.6B to 4B with no sign of saturating, but confirming the trend at production scale, and testing whether midtraining phases change persistence, requires pretraining budgets beyond ours. Our payload is also a fixed string. Whether adaptive payloads, such as a coding agent that inserts subtle vulnerabilities tailored to each codebase, can be planted as durably is an open and high-stakes question.

Takeaway. A few poisoned web documents, written as ordinary prose and amounting to 0.1% or less of the pretraining corpus, can make an agent trained with an entirely clean pipeline download and run an attacker’s script on command. The attack grows stronger with scale, leaves capability untouched, and can be triggered by an ordinary path in the user’s own request. The provenance of the pretraining corpus belongs in an agent’s security model, alongside its post-training and deployment safeguards.

Citation

@misc{hu2026agentic,
      title={Agentic Backdoors via Pretraining Poisoning},
      author={Xinyan Hu and Pingbang Hu and Chloe Loughridge and Xuanli He and Nicholas Carlini and Matthew Jagielski and Javier Rando and Yiming Zhang},
      year={2026},
      eprint={2610.37741},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2610.37741},
}