X-Ragent Teaches Chest X-Ray Agents When to Seek and Trust Evidence

Introduction

Interpreting a chest X-ray is not simply a matter of recognizing visual patterns. Depending on the task, a model may need additional evidence, such as a targeted auxiliary analysis or a quantitative measurement, and must determine whether that evidence is actually relevant to the case. Tool-augmented medical agents can provide these capabilities, but they do not always know when an extra tool call is necessary. They may also allow an inapplicable measurement to influence the diagnostic conclusion.

X-Ragent, proposed by a Carnegie Mellon University research team, is designed to address both issues. Its goal is not merely to give a model access to more tools, but to teach it when to request supporting evidence and when a measurement should be admitted into the reasoning process.

Core approach

• Separate evidence seeking from measurement admission. X-Ragent treats auxiliary-evidence retrieval and quantitative measurement acquisition as distinct capabilities. The policy learns whether additional evidence is needed, while a validity gate controls whether a measurement can be used.
• Initialize the agent with structured trajectories. The researchers build Rad-CoT, a 7.7K-sample corpus that combines causally ordered tool-use trajectories with report-style diagnostic reasoning. The corpus is used for agentic supervised fine-tuning of Qwen3-VL-8B.
• Use accuracy-gated reinforcement learning. The policy is subsequently optimized with Group Relative Policy Optimization. Process-level rewards for measurement consistency and auxiliary-tool cost are applied only when the final diagnosis is correct. This prevents a plausible-looking reasoning process from compensating for an incorrect answer.

Results and implications

Across three chest X-ray benchmarks, X-Ragent reports the strongest performance in the study. On the fully held-out ChestAgentBench, it delivers an 8.6-point improvement. The result highlights a broader design principle for medical agents: evaluation should consider not only whether the final diagnosis is correct, but also whether the model sought appropriate evidence and used measurements that matched the task.

The framework’s main contribution is therefore selective tool use rather than tool accumulation. A smaller set of relevant evidence may support a more faithful reasoning chain than a collection of indiscriminately generated measurements. Accuracy gating also reduces the risk that reinforcement learning rewards a polished but incorrect process.

The supplied material does not include detailed per-benchmark metrics or evidence of deployment in real clinical workflows. X-Ragent should consequently be understood as a methodological advance in evidence-grounded medical agents, not as a clinically validated diagnostic system.

Source: Hugging Face Daily Papers