Deleting Less Than 0.05% of Weights Can Still Post-Train an LLM

Introduction

Post-training a large language model usually means modifying many parameters. Full fine-tuning can be effective, but it creates large checkpoints and may overwrite capabilities learned during pretraining. A paper highlighted by Hugging Face Daily Papers examines a more radical alternative: instead of adding updates or continuously shifting many values, can a model gain a new capability simply by turning off a very small number of existing weights?

The authors’ answer is surprisingly positive. Their method, Deletion Tuning, reportedly reaches full fine-tuning-level performance while zeroing fewer than 0.05% of the pretrained weights. The result does not mean that the model is generally compressed by removing a large portion of its network. Rather, it suggests that a carefully selected set of parameter deletions can redirect behavior after pretraining.

How the method works

Deletion Tuning learns a binary mask over the gate projection of Transformer blocks. The original model weights are not replaced with a new dense checkpoint. During optimization, an annealed sigmoid mask is used so that the learned selection gradually moves from a trainable continuous form toward a discrete keep-or-delete decision. At deployment or sharing time, the main additional artifact is therefore a bitmask indicating which weights should be zeroed.

The reported evaluations cover three post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. Across all three, deleting the selected weights reportedly matches full fine-tuning at scale. The finding is especially notable because the method does not introduce a separate low-rank update or a dense set of learned weight deltas.

Key findings

• A much smaller update. The learned mask is hundreds to thousands of times smaller than the corresponding full-fine-tuning checkpoint, according to the abstract. This could make specialized model variants easier to store, distribute, and manage.
• Less forgetting. In the reinforcement-learning setting, prior-task accuracy remains close to the base model’s level. Full fine-tuning, LoRA, and random sparse fine-tuning reportedly lose as much as 5.4 percentage points in the same comparison.
• A structured, rather than random, change. The authors analyze stable rank, spectrum drift, and principal-weight overlap. On these measures, Deletion Tuning is closer to full fine-tuning than LoRA or random sparse fine-tuning.

Why it matters

The study challenges a common assumption about post-training: that useful adaptation requires changing a substantial fraction of a model’s parameters. If the result generalizes, a model developer could distribute tiny masks for specialized behaviors while keeping one shared base model. That would also be relevant to continual learning, where preserving earlier capabilities is often as important as acquiring new ones.

The evidence should nevertheless be interpreted carefully. The supplied material does not establish whether a mask transfers across model sizes or architectures, nor does it provide the complete model, dataset, and training details needed to judge how broadly the result applies. The strongest current conclusion is narrower: parameter location and structural role may matter more than the raw number of changed weights. Deletion Tuning offers a compelling hypothesis that post-training can sometimes be achieved by finding a few critical switches and turning them off, rather than rewriting the model broadly.

Source: Hugging Face Daily Papers