SoL-Pi Cuts Token Use for Long-Horizon Coding Agents

Introduction

Coding agents are moving beyond one-shot completion toward unattended exploration that may run for hours. Such agents repeatedly reason, invoke tools, inspect feedback, edit files, and test their results. In these long trajectories, the underlying model is only part of the efficiency equation. The harness surrounding it determines how actions are executed, how much context is retained, and how observations are presented back to the model. Poor orchestration can therefore turn every additional step into unnecessary token usage and API expense.

SoL-Pi focuses on this harness layer. The paper adopts an approach inspired by recursive self-improvement (RSI): automated research loops evaluate harness rollouts in an increasingly broad and diverse set of environments, then retain changes that appear reusable beyond the setting in which they were discovered. The objective is not to train a new foundation model, but to improve how an existing model conducts long-running work.

Key points

• The harness is the optimization target. SoL-Pi changes the agent’s operating process, including tool execution, context management, and information intake, rather than claiming a new model architecture.
• Four mechanisms survive selection. The resulting system combines changes in action execution, context compaction, observation handling, and delegated reading. Together, these address how an agent acts, what it remembers, how it interprets feedback, and which reading tasks can be assigned to a separate procedure.
• Search is performed across environments. Instead of tuning a harness only for one development setup, the auto-research loop tests rollouts in more varied environments. This is intended to favor improvements that transfer rather than brittle, local tricks.
• The reported efficiency gains are substantial. Across 51 EdgeBench tasks, SoL-Pi is described as achieving performance comparable to Pi while reducing recorded token traffic by 44.7%–49.0% and API cost by about one third. The paper estimates hourly savings of $8.75–$13.50 against native Codex and Claude Code harnesses, and $4.36–$5.71 against Pi.

Why it matters—and what remains unclear

The work points to a broader shift in agent engineering. Efficiency does not always require a larger or more capable model; it can also come from using the model more selectively. Removing redundant context, unnecessary observations, and avoidable reading can improve both cost and the number of trajectories that a system can run. If the discovered harness changes transfer across environments, one experiment could become a reusable piece of agent infrastructure rather than a one-off optimization.

The available source, however, is an abstract-level description. It does not explain the exact algorithms behind the four mechanisms, the baseline configurations, the variance of the results, or the composition of EdgeBench. “Comparable performance” should therefore be read as an aggregate claim from the paper summary, not as evidence that every task benefits equally. Deployment would also need to test whether compression removes important details, whether delegated reading adds latency, and how well the approach transfers across models and toolchains.

SoL-Pi’s main contribution is thus broader than token savings. It treats the agent harness as a system component that can be researched, selected, and improved continuously. That framing could become increasingly important as coding agents move from interactive assistants to persistent, autonomous workers.

Hugging Face Daily Papers