The Missing Step in LLM Math Reasoning May Be Discovery

Introduction

Large language models can solve an impressive range of difficult mathematics problems, but successful answers do not necessarily prove structural understanding. A model may produce a correct derivation by recognizing familiar patterns, or it may fail simply because it never identifies the right theorem or mathematical relationship. The paper The Missing Primitive studies this gap by looking beyond final-answer accuracy.

Four dimensions of mathematical reasoning

The authors introduce the idea of a Mathematical Primitive: a reusable mathematical structure, operation, or piece of knowledge that can guide problem solving. They use this idea to build PRIM, a benchmark that separates reasoning into four dimensions:

• Discovery: identifying the relevant theorem, relationship, or solution structure;
• Generation: turning that structure into useful mathematical expressions or steps;
• Digestion: understanding and absorbing an existing derivation;
• Execution: carrying out calculations, transformations, and subsequent reasoning accurately.

This decomposition matters because a single score can conceal very different capability profiles. Two models may achieve similar solution accuracy while differing substantially in their ability to find a solution strategy or execute one that has already been supplied. The study also reports that explicitly providing mathematical primitives can unlock considerable latent execution capacity. Some failures, therefore, may reflect missing structural guidance rather than an inability to perform the required calculations.

Discovery is the central bottleneck

The paper’s systematic diagnosis identifies Discovery as the dominant limitation in mathematical reasoning. Models can often continue a derivation once the relevant formula, theorem, or intermediate structure is available. They are less reliable when they must independently determine which structure applies to an unfamiliar problem.

This observation offers a useful explanation for why longer chains of thought do not automatically solve every mathematical failure. If the initial framing is wrong, additional steps can simply extend the wrong path. Improving reasoning may require better selection of the starting primitive, not merely more detailed continuation.

The authors also find that failures limited by Discovery are particularly amenable to repair. This suggests a more targeted post-training strategy: first identify which primitives a model is missing, then train it to discover and invoke those structures instead of applying uniform pressure to every complete solution trace.

From diagnosis to repair

Building on this analysis, the paper introduces ABSORB, a primitive-privileged self-distillation framework. Its central idea is to selectively transfer primitive-guided reasoning into a student model. The student is trained not only on the final answer, but also on the structural choices that make the answer possible.

According to the paper, ABSORB consistently improves mathematical reasoning over baseline methods across model scales and challenging benchmarks. The supplied material does not include specific numerical results, so the main contribution should be understood as a framework and a diagnosis of where improvement may be most effective.

Why it matters

PRIM provides a more granular lens for evaluating mathematical ability than answer accuracy alone. It can help researchers distinguish failures of structure discovery from failures of execution, while ABSORB points toward post-training organized around capability gaps.

The work does not establish that every mathematical task follows the same four-part pattern, nor does it settle what it means for a model to genuinely understand mathematics. Its contribution is more practical: it turns a broad question about mathematical reasoning into a set of components that can be measured, analyzed, and potentially repaired separately.

Source: Hugging Face Daily Papers