Why LLM Agent Backdoors Can Survive Post-Training

Introduction

Developers often turn third-party foundation models into software-engineering agents through supervised fine-tuning (SFT) and task-level reinforcement learning (RL). These stages are intended to improve coding, debugging, and tool-use capabilities. A new study highlighted by Hugging Face Daily Papers shows that the same process should also be viewed as a supply-chain security boundary: if a model already contains a hidden backdoor, benign post-training may not remove it completely.

In this setting, a backdoor is a concealed behavior activated by a particular input pattern. When the trigger appears, the model produces an attacker-chosen malicious output. The researchers study how such behavior changes as an inherited model passes through SFT and subsequent RL for software-engineering tasks.

Key findings

• SFT weakens but does not guarantee removal. Benign supervised data substantially lowers backdoor attack success, but some residual behavior can remain.
• RL can preserve the remainder. After SFT, task-level reinforcement learning often maintains the surviving backdoor and, in some cases, increases its attack success.
• Persistence depends on more than initial visibility. The analysis points to initial backdoor strength and gradient compatibility with benign training as important factors. A behavior that is strong enough and does not strongly conflict with normal learning may be harder to erase.
• Persistence can be optimized in advance. The proposed PersistBD refines an already-backdoored model before release, helping it withstand the downstream training process.

On Qwen2.5-Coder-7B, PersistBD increased attack success after SFT from 20% to 74%. After SFT followed by RL, it raised the figure from 20% to 76%, while maintaining comparable benign task performance. The result is important because it shows that persistence does not necessarily require an obvious loss on normal benchmarks.

Why it matters

The study’s broader contribution is to challenge the assumption that post-training automatically sanitizes a model. Developers may use their own data, improve task performance, and still inherit a behavior that was not visible in ordinary evaluation. For an agent that writes code, invokes tools, or changes project files, a hidden trigger can have more direct consequences than a malicious response from a conventional chat model.

Model providers should therefore treat pre-release backdoor screening and behavioral auditing as part of the delivery process. Model adopters should not use SFT or RL as a substitute for security validation. Evaluation should be repeated across training stages and should include trigger-oriented testing, behavioral comparisons, and monitoring of unusual tool-use patterns.

The supply-chain lesson is straightforward: an attacker can anticipate how downstream developers train a model and optimize the malicious behavior for that process. Agent builders need provenance checks, staged red-team evaluation, and post-deployment monitoring when adapting third-party weights. PersistBD does not merely describe a new attack technique; it illustrates why inherited backdoors must be treated as a lifecycle risk rather than a problem that training will automatically solve.

Source: Hugging Face Daily Papers