Learning More with Less: Uncertainty-Guided Compression for Reasoning Post-Training

Introduction

Reasoning post-training has created a growing efficiency problem for large language models. Larger reasoning corpora can provide more supervision, but they also increase compute and training time, often without proportional gains. The key question is therefore not simply how much data a model can consume, but which parts of a reasoning corpus are actually worth optimizing on.

In Learning More by Training Less, a Nanjing University team introduces an uncertainty-guided progressive contraction framework. Its central idea is to concentrate both supervision and optimization on reasoning trajectories that are most informative for the student model.

How the framework works

Many post-training pipelines treat a teacher-generated chain of thought as one fixed, complete path. That representation hides the uncertainty inside the reasoning process. A model may be confident at one intermediate step but face several plausible continuations at another. A single sampled chain cannot reveal this structure.

The proposed method builds Monte Carlo reasoning trees around intermediate reasoning states. By sampling multiple continuations from these states, it exposes process-level variation and constructs an uncertainty profile for the supervision. The resulting signal is used in two successive stages:

• Teacher-informed contraction: The original corpus is reduced to a compact pool of trajectories selected with help from the teacher-side uncertainty information.
• Learner-aware contraction: During supervised fine-tuning, an online routing mechanism adjusts which examples receive optimization according to the student’s current learning state.

The first stage answers what should remain in the training pool. The second answers what should be emphasized at a particular point in training. This makes the procedure different from static dataset pruning, where all selection decisions are made before fine-tuning begins.

Results and implications

According to the paper’s abstract, experiments span multiple reasoning benchmarks and LLM families. In some configurations, the method uses only 0.22% of the optimization budget of full-data SFT, corresponding to more than 450 times less training exposure, while maintaining or improving reasoning performance. The authors also report generalization across heterogeneous student families, black-box teachers, and weak-to-strong transfer settings.

The significance extends beyond a reduction in compute bills. The work suggests that reasoning post-training can be framed as an allocation problem: the objective is not to expose the model to every available trajectory, but to spend optimization steps where they are most likely to change the model. If process-level uncertainty can reliably identify valuable supervision, similar ideas could influence curriculum learning, data selection, and adaptive fine-tuning.

The reported efficiency should nevertheless be interpreted in context. The supplied material does not provide the full model configurations, dataset composition, retention ratios, or per-benchmark numbers. Therefore, the 0.22% figure represents a best-case result among the reported settings rather than a universal recipe. More testing is needed across tasks, teacher quality levels, and reasoning lengths. Even so, the paper offers a clear direction for efficient reasoning training: compress not only the dataset, but also the unnecessary optimization performed on it.

Source: Hugging Face Daily Papers