Salience Bias: Why LLMs Can Know Common Sense Yet Still Fall for Distractors

Lead

Large language models have become increasingly capable at formal reasoning, coding, and structured problem solving. Yet everyday commonsense reasoning remains a different kind of test. The paper “Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning” focuses on a subtle but important failure mode: models may become overly attached to explicit details in a prompt, even when those details are irrelevant to the real-world feasibility of the task.

The authors call this failure Salience Bias. In simple terms, an LLM may notice the most visible parts of a question—numbers, conditions, or instructions—and start complying with them, while overlooking implicit physical or commonsense prerequisites.

Key points

• Explicit cues can hijack reasoning: LLMs are trained to pay close attention to user-provided conditions. This is useful in many tasks, but in commonsense settings it can backfire. A model may perform unnecessary calculations or follow the surface framing of a problem before asking whether the situation makes sense at all.
• SaliTrap is designed to expose this weakness: The authors build the SaliTrap Benchmark, a dataset spanning four trap dimensions, to measure whether models can resist irrelevant but salient distractors.
• All tested mainstream models are affected: Across 12 state-of-the-art LLMs, the paper reports significant salience bias. The issue becomes more severe as distractor density increases. The study also finds that detecting a trap and actually avoiding it can be decoupled.
• The problem is often elicitation, not knowledge absence: By stripping away the misleading task framing and probing the models in a context-free way, the authors recover over 90% of sycophantic-compliance failures. This suggests that the required commonsense knowledge is often already present, but crowded out by salient distractors.
• Prompting can help without retraining: The paper further reports that lightweight inference-time prompting can substantially reduce the gap, indicating that better elicitation strategies may improve robustness.

Why it matters

This work reframes a common debate about LLM failures. When a model gives an absurd answer to a commonsense problem, the cause may not be a total lack of world knowledge. Instead, the model may be over-compliant with the prompt and overly focused on explicit details that should have been ignored.

For product teams, the implication is practical: prompts and interfaces should encourage models to check hidden assumptions before executing calculations or following surface-level instructions. For evaluators, SaliTrap highlights the need to test not only whether a model can produce correct answers, but also whether it can resist irrelevant information and change course after identifying a trap.

The broader lesson is that commonsense reasoning is not only about storing knowledge. It is also about reliably retrieving and applying that knowledge under misleading task conditions.

Source: Hugging Face Daily Papers