News · September 19, 2026 · opensource · 49 sections · ~3,078 words · 18 views

NVIDIA Open-Sources SoL-Pi: Autonomous Harness Optimizer Slashes Coding Agent Token Costs

Four AI-discovered mechanisms cut agent API spend by roughly a third with no task-quality penalty.

A team of NVIDIA, NTU, and MIT researchers published a paper and an open-source release on September 17, 2026, describing a system that lets AI agents automatically improve the infrastructure around themselves — not the model weights, not the training pipeline, but the harness layer that controls how a model talks to tools, manages context, and interprets environment feedback. The result, SoL-Pi, is a Pi agent framework extension that eliminates token waste through four mechanisms discovered autonomously by recursive self-improvement loops. On a 51-task held-out benchmark, it cuts token traffic by up to 49 percent and API spending by roughly a third compared with the Pi baseline, while retaining 93 to 94 percent of Pi's task performance.

The development matters now because API cost is increasingly the binding constraint on agentic coding deployments. As agents shift from single-shot completions to long, unattended trajectories — reading tool outputs, revising code, running tests, coordinating sub-agents — redundant work accumulates invisibly. Context grows. Large tool results replay into every subsequent request. Edits trigger predictable follow-up commands that each consume a separate model round trip. SoL-Pi's key finding is that these inefficiencies are discoverable by automation, and that fixes found by a self-improvement loop transfer reliably to unseen tasks and even unseen model backends.

What the Auto-Research Loop Actually Did

SoL-Pi's methodology begins with a question that has become more urgent as coding agents mature: before scaling recursive self-improvement, can an AI system first make the infrastructure for self-improvement cheaper to run?

The researchers built what they call a "Scaling Auto-Research Loop." A research agent observes execution traces of a separate coding agent running the base Pi harness, identifies recurring sources of overhead, and proposes candidate harness changes. Those candidates are then tested across a pool of executable environments — not a benchmark, but a training set of 535 environments the research team constructed from GitHub issue-pull-request pairs and synthetic tasks. Candidates that survive a capability floor (task quality stays within predeclared tolerances) and improve at least one efficiency metric are retained and hardened. Held-out results never feed back into the search; a failed evaluation on the reserved EdgeBench tasks rejects the candidate outright.

The search started with 152 proposed directions, organized into six hypothesis families: context, progress, tools, delegation, prompt and policy, and improvement and evaluation. Only four directions survived the full selection process — a roughly one-in-forty acceptance rate. Those four became SoL-Pi.

The orchestration architecture evolved through three designs before reaching what the paper calls "disposable skill loops" — an approach in which each research experiment instantiates a fresh copy of a minimal template, runs to completion, and discards its modified orchestration code. That design allows hundreds of parallel search lineages without the coordination overhead that accumulated in earlier compiled-workflow and code-orchestration approaches.

The Four Mechanisms SoL-Pi Discovered

Each of the four surviving mechanisms targets a different, complementary source of waste in the agent-environment interaction loop.

Action Fusion addresses a recurring sequence in base Pi trajectories: after editing a file, the agent typically issues a separate command to test, build, or run the changed code. That sequence requires an intermediate model round trip — the model waits for the edit to return before deciding to issue the follow-up. Action Fusion extends Pi's tool schema to expose a fused action that combines the edit with an optional follow-up command, returning both outcomes in a single observation. The research agent found this pattern in 12.3 percent of cross-turn transitions in the training trajectories. Under full triggering, the Oracle Analysis projected an 11.5 percent reduction in total tokens, and a 10.8 percent reduction in model turns, for the evaluated trajectory set.

Online Context Compact changes when the harness decides to compress the agent's accumulated context. The native Pi approach defers compaction based on a global token limit. Online Context Compact instead uses plan-step completion as the trigger: at each point where the agent finishes a subtask, the harness estimates the expected savings from compacting against the cost of rewriting the prompt cache prefix, and fires compaction only when the math clears a predeclared gate. This prevents both premature compaction (which wastes the cache rewrite cost) and late compaction (which lets the context grow unchecked). In the full SoL-Pi stack on GPT-5.6 Sol, Online Context Compact is the single largest contributor to cache-read token reduction.

ObservationPack solves the replay problem for large tool outputs. In base Pi, a file read or a large command result reappears in full in every subsequent model request — occupying both context and the prompt cache even when only a few lines remain relevant to the current step. ObservationPack locally archives results exceeding 10 KiB and sends them in full for the first two provider requests. From the third request onward, it replaces the payload with a stable handle, the original size, and a short excerpt of head and tail lines. The agent can retrieve exact pages through the handle when it needs them. Smaller results are unaffected. The mechanism's research lineage traced back to context hypothesis families C23 and C24 — ideas about building observation packs before paying for full content — but its implementation ended up modifying the observation boundary rather than the context layer, illustrating how the search can take unexpected routes from hypothesis to mechanism.

Evidence-Preserving Reducer targets the cost of reading long diagnostic logs. Build outputs and test logs can run to thousands of lines, but only a handful typically determine what the agent does next. The reducer sends qualifying logs (build and test results of at least 4 KiB) to a secondary, lower-cost model — the researchers used GPT-5.6 Luna — which extracts a compact receipt listing key evidence: exit status, exact quoted lines, source hash, and size. A deterministic verifier then checks every field of that receipt against the archived original. If verification fails, credentials are suspected in the output, or the receipt provides no size reduction, the harness falls back to the full original log. The frontier agent sees only the verified receipt in its context; it retains full responsibility for diagnosis and action selection. File reads and search results bypass the reducer entirely, preventing it from discarding information the agent has actively sought out.

The four mechanisms are designed to be complementary. ObservationPack becomes more selective when Evidence-Preserving Reducer is also active, because the reducer's verified receipts are smaller and already marked, so ObservationPack recognizes and skips them. The full stack achieves a larger token-efficiency gain than the sum of the individual mechanisms on each mechanism's triggered tasks — a pattern the authors describe as consistent with complementarity, while noting that the comparisons are descriptive rather than isolated.

Why Harness Optimization Is Distinct From Model Optimization

It is worth being precise about what SoL-Pi does and does not do, because the terminology of "self-improvement" carries a lot of freight.

SoL-Pi does not modify model weights. It does not adjust training data, change post-training procedures, or alter how a foundation model reasons. What it modifies is the harness: the software layer that tells the model which tools to call, how to manage growing context, how to package and interpret environment observations, and when to delegate sub-tasks to cheaper secondary models. A harness sits between the model and the environment; it shapes what information the model sees and in what form, but it does not change the model's internal computations.

Prior work in agentic efficiency has largely focused on the other layers — lower-cost per-token inference through faster attention kernels, smaller memory footprints through KV cache compression, and model-level changes like sparse activation in mixture-of-experts architectures. SoL-Pi's contribution is orthogonal: it reduces the number of tokens generated and processed at the harness layer, without requiring any of those model-level changes. A team running GPT-5.6 Sol or Opus 5 behind a Pi harness can install SoL-Pi and capture the efficiency gains without touching the model serving stack at all.

That orthogonality is significant. It means the harness-layer and model-layer efficiency improvements can stack. A team that adopts hardware with better inference efficiency alongside a harness that reduces the number of requests and their size gets gains from both. The research question the SoL-Pi team is pursuing — can RSI systematically find those harness-level gains at scale — is separate from the race for faster inference hardware or cheaper model weights.

What EdgeBench Measures and How SoL-Pi Performed

EdgeBench is a ByteDance Seed benchmark of 134 real-world, day-scale tasks spanning scientific and ML research, systems and software engineering, optimization, formal verification, and game-playing domains. The benchmark was designed specifically to measure long-horizon agent performance — tasks run for up to 12 hours of continuous interaction — making it one of the few public evaluations where harness-level efficiency differences can actually accumulate and be measured. SoL-Pi used only the 51 publicly available tasks, reserving 40 of those for final evaluation and using 11 for one-way acceptance of frozen candidates.

All results below are paper-reported; independent third-party reproduction of SoL-Pi's EdgeBench figures had not been published as of September 19, 2026.

The paper reports two SoL-Pi operating points. The Efficiency point runs all four mechanisms. On GPT-5.6 Sol, it produces 1.10 billion total recorded tokens against Pi's 2.15 billion — a 49.0 percent reduction — at an API cost of $894 versus Pi's $1,339, a 33.2 percent cost reduction. Average task score is 42.003 versus Pi's 44.833, retaining 93.7 percent of Pi's performance. Against the native Codex harness (which scores 34.738 at a cost of $1,787), SoL-Pi [Efficiency] is both cheaper by half and higher-scoring by a material margin. Against the native Claude Code harness running Opus 5 ($2,535 for a score of 43.689), SoL-Pi [Efficiency] on Opus 5 costs $1,158 — 54.3 percent less — at a score of 42.224, slightly below Claude Code's native score.

The Performance point selects the single mechanism with the highest average score per backend (ObservationPack for GPT-5.6 Sol, Action Fusion for Opus 5). On Sol, this raises average score from Pi's 44.833 to 47.208 — a 5.3 percent gain — while reducing token traffic by 6.1 percent and improving token efficiency by 9.8 percent. On Opus 5, Action Fusion alone raises average score from 44.756 to 50.482, the highest score of any evaluated configuration.

The transfer result is notable. SoL-Pi was discovered and tuned entirely on GPT-5.6 Sol trajectories. When applied without modification to Opus 5, it retains 94.3 percent of Pi's average score on that backend while reducing token traffic by 44.7 percent and cutting API cost by 33.5 percent. Mechanism activation rates are lower on Opus 5 than on Sol — the model's behavioral patterns differ, so the harness fires less often — but every mechanism still improves token efficiency on the tasks where it activates. That cross-model generalization distinguishes SoL-Pi from prior harness-optimization work where improvements on the training model often failed to transfer.

Beyond EdgeBench, the paper reports Terminal-Bench 4 results on 63 CPU-only tasks. SoL-Pi solves 15, compared with Pi's 18 and Codex's 18 — three fewer completions. However, SoL-Pi's total model cost across those tasks is $211.12, versus $286.45 for Pi and $272.35 for Codex, a 26.3 percent cost reduction versus Pi. On the IMO 2026 formal math evaluation (six problems requiring Lean 4 verification), SoL-Pi passes three problems at a total cost of $62.69, the lowest cost per passed problem at $20.90 versus $25.32 for Pi and $22.89 for Codex.

The Terminal-Bench 4 task-completion gap is worth flagging. Three fewer solved tasks represents a real capability cost for workflows where completion rate matters more than cost. The paper acknowledges the tradeoff without resolving it — SoL-Pi's Efficiency stack trades some completion probability for a substantial cost reduction, while the Performance stack avoids most of that tradeoff by using a single mechanism rather than all four.

Where SoL-Pi Sits Relative to Prior Harness Optimization

Harness optimization has become an active research area alongside model capability improvements. Several systems preceded SoL-Pi and inform its design, including Meta-Harness (which searches over executable harness programs using prior code, scores, and traces while maintaining a Pareto frontier over task performance and context cost), Recursive Harness Self-Improvement (which refines prompt-level specifications per task), and Agentic Harness Engineering (which evolves modular harness components from trajectory evidence). All three approaches are discussed in the paper, which situates SoL-Pi's contribution relative to each.

A 2026 study by Wang et al. is especially relevant context: that work found that evolved harnesses often overfit their search tasks, delivering only marginal improvements on held-out evaluations when search and evaluation data overlapped. SoL-Pi's design is a direct response. The broad-to-deep search funnel keeps final evaluation fully isolated from all development work; held-out trajectories never enter subsequent analysis; and a failed held-out validation rejects the candidate without reopening the search. The diversity of the 535 training environments — spanning real GitHub issue-PR pairs across many repositories, plus synthetically generated verifier-driven tasks — was chosen specifically to reduce task-specific overfitting.

The efficiency objective also shapes what survives. Optimizing for task quality alone creates an obvious path to benchmark gaming: the harness can encode patterns specific to training tasks. Optimizing for token efficiency at constrained quality targets work that recurs across tasks. Removing repeated context, combining actions that don't require intermediate decisions, and avoiding re-transmission of large results are properties of the interaction loop itself, not of any particular task. They have a better chance of generalizing. That is the argument the paper makes, and the cross-model transfer results provide preliminary support for it.

Agent Swarms and the Infrastructure Argument

The paper includes an agent swarm experiment that extends the single-agent results into a collective setting. A coordinator running GPT-5.6 Sol (via Codex) directed 20 GPT-5.6 Luna workers across five groups of four in a two-hour kernel-optimization task, scored in simulated machine cycles. Three configurations were compared from the same frozen starting point: a single coordinator agent, a coordinator with 20 Pi-baseline workers, and a coordinator with 20 SoL-Pi workers.

The SoL-Pi swarm reached 1,127 cycles at a cost of $60.11 — 17.5 percent fewer cycles and 26.8 percent lower API cost than the Pi baseline swarm (1,366 cycles, $82.12). Both passed all eight speed thresholds; the Pi baseline swarm missed the final threshold. The single coordinator agent remained least expensive overall ($39.20) but finished at 1,333 cycles, between the two swarm results.

The swarm result matters for a specific reason: multi-agent coding workflows are increasingly common in production, and API cost scales with agent count. A harness that reduces per-agent token use by roughly 45 percent could make swarm deployments that are currently cost-prohibitive viable. The paper frames this as a demonstration of collective-search pattern: independent agent trajectories, selective evidence sharing, and verification-gated acceptance of best results.

Limitations and What Independent Evaluation Will Need to Address

Several important caveats belong alongside the reported numbers.

All benchmark figures come from the paper's authors. Independent replication of the 44.7–49.0 percent token reduction on EdgeBench had not appeared as of the arXiv listing date. EdgeBench itself has not published a mechanism-by-mechanism cost analysis; the paper's reported token counts are based on recorded API traffic during evaluation runs, which the paper distinguishes from full API equivalents.

The cost figures use API list prices from August 17, 2026. API pricing changes, and the hourly savings estimates ($8.75–$13.50 versus native Codex and Claude Code harnesses) will not remain accurate as underlying model prices evolve.

The Terminal-Bench 4 completion shortfall — 15 tasks solved versus 18 for Pi and Codex — is a concrete performance cost that the paper does not fully explain. The authors note that GPU-dependent tasks were excluded from the evaluation due to infrastructure limits, which may have affected the task set in ways that are difficult to disentangle from the harness effect.

Mechanism transfer to Opus 5 reduced trigger rates compared with the training backend. The paper attributes this to the harness being optimized on Sol trajectories — the model's behavioral patterns differ, so activation is less frequent. The authors list multi-backend training as a future direction. That gap matters for teams using Claude Code or other Opus-based configurations: the efficiency gains are real but smaller, and the mechanism may not fire at all on some task types.

The paper's authors also flag a deeper limitation: running complete auto-research loops is computationally expensive, and controlled comparisons of search breadth and depth under a fixed budget remain unsolved. The 150 proposed directions and 500+ environments represent a large experiment; scaling further without establishing search laws remains a research challenge.

What the Recursive Efficient Improvement Concept Points Toward

The most consequential implication in the paper is not the four specific mechanisms but the research direction the team calls "recursive efficient improvement." Their plan is to use SoL-Pi as the starting harness for the next auto-research cycle. A more efficient harness reduces the per-run cost of every search iteration, which means a fixed compute budget can cover more candidate directions, more executable environments, and more refinement iterations than the original search could. In that view, the efficiency gain from SoL-Pi is not just an outcome — it is a resource for the search that builds its successor.

The authors frame this as a long-term research vision, not a compounding effect demonstrated by the current work. But it addresses a structural constraint on RSI research: the auto-research process is itself expensive, which limits how aggressively it can be scaled. If each generation of harness lowers the cost of running the next generation's search, the constraint relaxes incrementally. That is a different kind of AI efficiency story than the ones dominated by faster attention kernels, better quantization, or lower-cost inference hardware — and it operates at a layer that any team using a compatible coding agent framework can access today.

SoL-Pi installs on top of an unmodified Pi release with a single command (pi install git:github.com/NVlabs/SoL-Pi), requires Node.js 22.19 or newer and @earendil-works/pi-coding-agent version 0.85.1, and carries an MIT license. All four mechanisms are opt-in and disabled by default. The code is openly available at github.com/NVlabs/SoL-Pi under a 2026 NVIDIA copyright, and the search methodology that produced it represents as notable a contribution as the four mechanisms themselves: a working template for how to run RSI at the harness layer at scale.