Opening the Black Box: Extracting Hidden Reasoning from Frontier Models

Introduction

The recent progress of frontier language models is often explained by stronger reasoning. Yet for closed systems, observers cannot inspect the original chain of thought and must infer the process from final answers and benchmark scores. The paper Capable yet Parsimonious explores a practical way to observe more of that process: a custom tool registered through a standard API feature can prompt models to externalize parts of their intermediate reasoning.

The study covers open-source models as well as closed frontier systems including GPT-6 Astra, GPT-5.6 Sol, Claude Opus 4.8, and Claude Sonnet 5. Rather than assuming that every exposed trace is authentic, the authors first compare the extracted reasoning with native chain-of-thought in open models. This calibration matters because a model might generate a plausible explanation after arriving at an answer, effectively rationalizing its output.

Key findings

• The extracted traces are behaviorally useful. On competition mathematics, science, and code-generation tasks, their performance aligns with native reasoning and substantially exceeds no-reasoning baselines.
• Models organize reasoning differently. The analysis examines token efficiency, reasoning-step categories, and induced reasoning trees to compare how systems expand, compress, and structure intermediate work.
• Astra favors directed reasoning. It appears to select a correct trajectory earlier, backtrack less, and leave elementary calculations implicit while externalizing only crucial steps.
• A short trace is not conclusive evidence. Genuine competence, memorized data, and post-hoc explanation can all look like a fast path to a correct answer from the outside.

Why it matters

This work shifts attention from how often a model is correct to how its observable reasoning is organized. If tool-induced traces can be captured reliably, they may help researchers study where models change direction, how they recover from errors, and how different systems trade reasoning quality against token cost. That could support more informative evaluations, monitoring, and safety audits.

The limitation is equally important: an exposed text trace remains model-generated behavior, not a complete and necessarily faithful record of internal computation. Further testing across tasks and adversarial settings will be needed to determine whether such traces consistently reveal failure mechanisms or merely produce convincing explanations. The authors state that the method has been disclosed to OpenAI and Anthropic.

Source: Hugging Face Daily Papers