Evidence-RL Pushes VLMs to Answer from Visual Evidence, Not Shortcuts

Introduction

Vision-language models can often answer image questions convincingly, but a correct answer does not always mean the model has used the right visual evidence. In many cases, a model may rely on language priors, dataset shortcuts, or irrelevant contextual cues in the image. Evidence-RL: Towards Evidence-intensive Visual Reasoning focuses on this gap between being right and being grounded.

The paper proposes Counterfactual Evidence Disentanglement (CED), a training-time audit designed to check whether a sampled answer actually depends on the image region that should support it. Instead of only encouraging models to attend to images globally, CED asks a more causal question: if the key evidence region is neutralized, does the model’s support for the answer drop more than it would for comparable non-evidence regions?

Key Ideas

• Grounding over surface correctness: The method targets answers that are not only correct, but correct for the right visual reason.
• Object-centric evidence auditing: For each response, CED identifies an Evidence Region using weak object-level proposals and neutralizes that region during training.
• Counterfactual comparison: The resulting support drop is compared against matched non-evidence regions, helping distinguish true evidence reliance from generic sensitivity to visual perturbation.
• Reward design inside GRPO: The evidence signal is combined with answer correctness in a GRPO-style reinforcement learning setup. Correct answers are rewarded more when they depend on the intended evidence path rather than shortcuts or nuisance cues.
• No inference-time cost: CED is used during post-training. The paper states that it requires no question-specific evidence annotations and adds no overhead at inference time.

Why It Matters

The important shift here is from measuring whether a VLM uses an image at all to checking whether its answer causally depends on the right part of the image. Prior perception-aware post-training methods often use global perturbations or attention-based proxies. Those signals can be useful, but they may not prove that a particular answer is supported by the local evidence that matters.

CED reframes grounding as an evidence audit. If removing a candidate evidence region weakens the model’s answer more than removing similar non-evidence regions, the answer is more likely to be tied to the correct visual path. This is especially relevant for fine-grained visual question answering, scene understanding, and other settings where background correlations can easily mislead a model.

According to the authors, Evidence-RL improves over previous RL-based post-training methods across nine public benchmarks and four backbones, with targeted analyses supporting the object-centric nature of the signal. The approach also has practical appeal because it avoids extra question-level evidence labeling and does not change inference-time deployment.

A likely limitation is that the quality of weak object proposals can affect the audit: missed or poorly localized objects may reduce the reliability of the evidence signal. Still, the direction is clear. For multimodal reasoning systems, future training objectives may need to reward not just accurate outputs, but outputs that are demonstrably grounded in the image evidence.

Source: Hugging Face Daily Papers