“Wait” Was Inherited, Not Invented, by Reasoning Models

Introduction: a visible token is not a complete explanation

Reasoning models often place {wait} in an answer before revisiting an earlier step. Because the token resembles an explicit pause for self-correction, it has become associated with reinforcement learning, particularly training with verifiable rewards. The paper Inherited, Not Invented asks whether that association is causal: did RL teach models a new revision strategy, or did models inherit a familiar textual marker from their data and later training stages?

The authors study open models across their training histories. Rather than treating raw frequency as a proxy for reasoning, they sample model outputs and measure where the marker appears, what form it takes, and how it is used. They also introduce errors into problems and test whether the marker corresponds to an actual attempt to reconsider a mistake.

Main findings

• RL does not generally coin the marker. On base models that do not use {wait}, zero-RL training can still produce substantial MATH-500 accuracy gains while the marker remains nearly absent. Improved task performance therefore does not require the word.
• Training data is the main source. The marker appears during mid-training in the OLMo-3 history. Reasoning-trace fine-tuning also installs it in most tested models whose bases lacked it.
• RL recipes behave differently. When the base model already uses {wait}, the released RL-Zero recipes for OLMo-3 and MiMo multiply its occurrence. KL-free GRPO applied to the same base does not show the same effect.
• Form and behavior can diverge. OLMo-3’s recipe wears down the marker’s surface form while making it a more sensitive but more permissive alarm. RL after supervised fine-tuning changes the marker little, whereas the authors’ own GRPO recipe erases a marker seeded by fine-tuning.
• Automated revision judgments are fragile. Marker counts alone cannot demonstrate RL-induced reasoning. The paper also warns that LLM judges evaluating revision behavior may reverse trends found by human annotators.

Why it matters

The study challenges a simple reading of reasoning traces. A special token can be a convention inherited from the corpus, a pattern installed by supervised fine-tuning, or an output habit amplified by an optimization recipe. It is not automatically a transparent window into the model’s internal reasoning process.

For researchers, the practical lesson is to separate four questions: whether a model’s accuracy improved, whether a marker appeared, whether the marker was used in the expected position, and whether the model actually corrected an injected mistake. These measurements should be compared across the base model, intermediate checkpoints, supervised data, and RL objectives.

For users, {wait} can be treated as an observable formatting signal, not as a guarantee of reliability or deeper thought. More broadly, the paper argues for behavioral tests over stylistic proxies. A model may learn to display the language of reconsideration without becoming better at reconsidering, while another model may improve its reasoning without displaying the expected word.

Source: Hugging Face Daily Papers