Why the First Few Tokens Can Control a Model’s Chain of Thought

Introduction

Chain-of-thought monitoring is often proposed as a way to supervise increasingly capable AI agents. By inspecting a reasoning trace, an evaluator may detect whether a model is following an objective or developing an undesirable strategy. That approach depends on an important assumption: the model must be able to control what it verbalizes. A new paper by Hugh Van Deventer, Dmitrii Troitskii, and Laura Ruis argues that this ability may be constrained before the reasoning process has properly begun.

What the study found

The authors associate weak controllability with an entropy collapse in the first few tokens produced after an opening thinking tag. In practice, models strongly favor a small set of familiar openings. That early commitment can make it harder to follow later instructions about how the reasoning should be written or modified.

The study tests two inference-time interventions:

• Prefilling: supplying an initial token before generation begins. The results indicate that even one token can substantially improve controllability.
• Banning frequent openings: preventing common starting tokens and combining the restriction with in-context examples. Across the evaluated models, this raised average compliance by 34–41 percentage points.

The gains are not uniform across task types. Models respond more reliably when asked to add information, transform information, or adopt a particular style. Suppression tasks—avoiding or removing specified information—show more varied behavior. The authors therefore treat addition and style control as a group distinct from suppression control. Within each group, model performance tends to correlate positively, while correlations between the groups are mostly negative.

Why it matters

The findings suggest that chain-of-thought evaluations should separate difficulty in initiating a requested trace from a broader inability to alter reasoning content. Otherwise, a benchmark may mistake a bad opening for a general failure of controllability. The paper also reports that the connection between controllability and monitorability depends on the evaluation: addition and style control more often correlate positively with monitorability, while suppression can show negative associations.

For AI safety research, the first generated token may therefore be more than a formatting detail. It can act as a practical steering point for oversight experiments. Still, the results do not establish general, reliable control over reasoning traces. They show that targeted interventions can recover specific forms of compliance, while suppression remains less predictable. Future evaluations may need separate reporting for addition, style, and suppression tasks, as well as a clear distinction between starting and modifying a chain of thought.

Source: arXiv