Clinical LLMs Should Not Do the Math Themselves

Introduction

Clinical calculators turn patient data into scores, risk categories, or recommendations. Their formulas may be simple, but a single arithmetic mistake can change the result. Language models are particularly unreliable when they must carry out multi-step numerical operations in text. The conventional engineering response is to implement and validate every calculator as a dedicated function. This study examines a more general alternative: the model writes case-specific Python, while a restricted local executor performs the calculation deterministically.

Key findings

• Reasoning and arithmetic are separated. The model decides how to use the formula, selects values from the case, and produces code. The executor handles the numerical operations instead of asking the model to calculate in prose.
• The comparison controls information access. On MedCalc-Bench Verified, the researchers evaluate 1,100 cases across 55 calculators with Qwen2.5-7B and Qwen2.5-32B-AWQ. Both direct calculation and program solving receive the formulas, gold variables, and the full clinical note.
• The 7B result is inconclusive. Program solving scores 75.31%, compared with 72.02% for direct arithmetic, a paired gain of 3.29 points. The 95% calculator-cluster interval is [-3.49, 10.38], so the observed difference does not establish a reliable advantage.
• The 32B result is stronger. The larger model reaches 90.53% with program solving versus 83.47% with direct calculation, a 7.05-point improvement. Its interval, [0.47, 14.60], stays above zero.
• A fixed library trades coverage for certainty. The handwritten library is exact on its 440 supported cases, but abstains on the rest, producing 40.0% overall accuracy.
• The formula is part of the safety problem. An audit identified version, use, or coefficient concerns in 16 of the 55 benchmark calculators.

Why it matters

The study does not show that adding Python makes clinical language models reliable by itself. It shows that deterministic execution can remove one class of failure: arithmetic drift inside generated text. The model still has to identify the relevant patient variables, select the right calculator, respect units and conditions, and generate usable code. If extraction or formula selection is wrong, a deterministic executor will simply produce a precise answer to the wrong problem.

The model-size difference is also important. Under matched access to formulas, variables, and notes, the 7B model does not obtain a dependable benefit, while the 32B model does. This suggests that tool use is not independent of the model’s ability to decompose tasks, write valid code, and decide when execution is appropriate. The results do not establish that size alone explains the gap.

A robust clinical system therefore needs layered controls: clinician-reviewed and versioned formulas, a restricted execution environment, checks for units and missing values, validation of extracted variables, and a safe abstention path. Deterministic arithmetic is useful, but it is only one component of trustworthy clinical decision support.

Source: Hugging Face Daily Papers