The More Popular a Fact, the Harder It Is for an LLM to Forget

Introduction

Making a large language model forget a selected fact is more complicated than reversing the gradient associated with that example. Facts that appear repeatedly in pretraining data can become more deeply encoded than rare facts. As a result, applying the same unlearning pressure everywhere may remove obscure knowledge while leaving popular knowledge recoverable. The paper The More Popular, The Harder to Forget presents AdaPop, a method designed around this asymmetry.

How AdaPop works

The method treats forgetting as a fact-dependent optimization problem:

• Popularity-aware weighting: AdaPop assigns each fact a popularity-dependent exponent. The signal can be obtained from an external proxy such as Wikidata sitelinks, or estimated with an LLM-as-Judge setup.
• Local confidence: Fact-level popularity is combined with confidence at the token level. This lets the update respond to how the model actually expresses a target fact rather than relying only on a coarse sample-level weight.
• Automatic retain control: Forgetting can damage nearby capabilities if the update is too aggressive. AdaPop therefore uses a dual-ascent controller to adjust the retain penalty after each epoch, seeking a more stable forget-retain trade-off.

Reported results

The abstract reports evaluations across three model families and two benchmarks. Under paraphrased queries, AdaPop leaked roughly five times less forgotten content than competing methods. Under adversarial reformulations, it reduced leakage by about a factor of 1.6. These figures describe the paper’s evaluation conditions and should not be interpreted as a universal guarantee across all models, prompts, or datasets.

The authors also examine internal representations. After unlearning, hidden states for the forget set move farther from those of the pre-unlearning model under AdaPop than under the comparison methods. At the same time, representations for the retain set remain close to their original states. This supports the intended design: concentrate change around the targeted knowledge rather than broadly perturbing the model.

Why it matters

AdaPop highlights a useful shift in perspective. Unlearning may need to allocate different amounts of optimization effort according to how strongly a fact is embedded, rather than treating every deletion request as equally difficult. That could be relevant for privacy removal, data governance, and model maintenance.

There are also clear caveats. Popularity is only a proxy for memorization strength and may reflect visibility rather than actual importance. External sources and LLM-based judges can introduce their own bias. Robust evaluation should therefore include paraphrases, adversarial reformulations, multiple model scales, and measurements of retained capabilities—not merely whether a direct prompt stops producing the target answer.

AdaPop does not make unlearning a solved problem, but it offers a more targeted and feedback-driven direction for reducing residual knowledge leakage.

Source: Hugging Face Daily Papers