Scientific Agents Need More Than Correct Answers: Auditing Evidence and Execution State

Introduction

For a scientific AI agent, a correct final answer is not enough to establish reliability. An agent may reach the same verdict while using a different set of evidence, or it may make a plausible recommendation after a silent numerical error has altered an intermediate computation. BioAgent-DF examines this gap by auditing not only outputs, but also the evidence, tool state, and execution history behind them.

What the experiments found

The researchers stored paired reference and candidate values at the same locations in a scientific computation. They then restored or transplanted those values and ran the agent on healthy hardware. In SciFact experiments, Qwen3.5-4B and Gemma 3 4B IT showed repeatable changes in what they read or cited across several capture-and-claim combinations. Full restoration could recover the reference evidence sets while leaving the verdict unchanged. Conversely, verdict changes seen under a baseline prompt did not recur when the prompt included explicit examples.

These results separate several notions that are often treated as one: recovering numerical state, recovering a ranked list, and restoring the visible history available to the agent. An unchanged answer therefore cannot prove that the agent followed the same evidential path.

The more operationally important test used hERG cardiac-safety screening. Silent GPU faults produced incorrect numerical values without a crash, NaN, or infinity. Two compounds labeled as hERG blockers in the benchmark, K12252 and K1461, were assigned to advance without testing rather than receive patch-clamp testing. The aggregate result still looked stable: 351 of 402 compounds were labeled blockers. A count-only review would therefore miss the fact that the selected compounds and testing assignments had changed.

Basic checks for data integrity, finite values, and probability ranges detected none of the 16 fault sources. Checksums applied to linear layers detected all 16. On the measured workload, streaming verification increased native forward-pass time by 25.8%.

Recovery was also state-dependent. Replacing the prediction file after the agent had already read faulty values recovered none of the 39 changed decisions. Restoring the execution state saved before that read, then supplying certified scores, recovered all 39. No incorrect action was committed in the experiment.

Why it matters

The study suggests that scientific-agent audits should track two separate questions: What did the agent conclude, and what evidence and state led it there? Logs should connect tool inputs, intermediate values, ranking operations, citations, model versions, and actions. A corrected file may be insufficient if faulty data has already influenced memory or subsequent computation; recovery may require rollback and re-execution.

The findings do not imply that every agent or workload will fail in the same way. They do show why answer-only evaluation is too narrow for biomedical automation. Reproducibility must include decision provenance, not just the final text.

Source: Hugging Face Daily Papers