A coding agent working through the night does a lot of rereading. Every time it asks the model for the next step, the harness sends the conversation so far: the task, the files it opened, the long test log from twenty minutes ago. Most of that text has already done its job. It gets paid for again on every turn.
Most work on making agents cheaper attacks the price of each token: faster kernels, compressed models, cheaper models. A paper from NVIDIA, NTU and MIT, posted to arXiv on 17 September 2026, attacks the number of tokens instead. It does not touch the model. It changes the harness around it, and the changes were found by another AI running research loops.
The result, called SoL-Pi, cut recorded token traffic by 44.7 to 49.0% on a held-out benchmark across two frontier models. This post explains what a harness is, how the AI searched for improvements, the four changes that survived, and what the savings really compare against.
What is a harness?
A harness is the program that sits between a language model and the world. The model only reads text and writes text. The harness decides which tools the model can call, runs those tools, collects their output, decides what goes into the next prompt, and decides when to shorten the conversation. Claude Code, Codex and Pi are harnesses. The same model can run inside different harnesses: the paper ran GPT-5.6 Sol in both Codex and Pi, and Opus 5 in both Claude Code and Pi.
Think of the model as a skilled worker and the harness as their desk: which papers sit in front of them, which get filed away, which tools are within reach. Two desks can make the same worker fast or slow. The analogy stops working in one place: a person remembers yesterday, while a model only knows what the harness puts on the desk for this one request.
That makes the harness a real lever on cost. If it sends less text per request, or makes fewer requests, the bill drops with the same model and the same task. The hard part is that its pieces are tangled. The paper puts it this way: "a change that is locally beneficial may cause downstream failures or shift token costs to later stages of execution." Cutting a log short saves tokens now and can cost a failed fix later.
How did an AI improve the harness?
The authors used an autoresearch loop, a pattern Andrej Karpathy published as a short instruction file for agents: propose a change, implement it, run a fixed experiment, look at the result, keep it or throw it away, repeat. They added two steps. The implementer keeps retrying until an explicit completion criterion is met (a pattern known as the Ralph loop), and a separate reviewer agent checks the change before it is tested.
The research AI did not guess blindly. It read execution traces (full records of what the agent did on a task, request by request) from an agent running the base harness, Pi, and looked for waste: repeated actions, context that kept growing, huge tool outputs, logs that said little.
Two rules kept the search honest. First, the scoring rules were fixed before the search began and kept out of the optimizer's reach. A candidate passed only if every capability score stayed within a set tolerance and at least one efficiency measure improved. Second, the final benchmark, EdgeBench, was held out. It was used only on frozen candidates (11 of its 51 public tasks to accept or reject them, the other 40 for the final result), and "Held-out results never feed back into the Auto-Research Loops". This matters because an earlier study found that harnesses evolved this way tend to overfit the tasks they were tuned on and gain little on new ones.
The scale was large. The search started from 152 proposed directions and ran in 535 environments: 495 built from real GitHub issues and the pull requests that fixed them (with the fix and its test hidden from the agent), and 40 synthetic tasks with automatic checkers. The paper reports more than 3,000 runs and more than 60,000 agent-environment interactions.
What did it find?
Four mechanisms survived. None of them is exotic. Each removes one specific kind of repeated work.
- Action Fusion: an agent often edits a file and then, in a separate request, runs the test. Action Fusion lets the edit tool take a follow-up command and returns both results together, so three API calls become two. Commands that need to look at the edit first stay separate.
- Online Context Compact: compaction means replacing the long conversation with a shorter summary. It saves input tokens on every later request, but it breaks the prompt cache (the provider's discount for resending the exact same opening text), and rewriting that cache costs money. This mechanism only considers compacting when a step of the agent's plan finishes. It estimates how many requests are left and compacts only if the projected savings beat the cache rewrite. It also compacts when the context window is nearly full.
- ObservationPack: a tool output over 10 KiB is saved to disk. It is sent in full for the next two requests, then replaced by a handle, its size and a short excerpt of the first and last lines. If the agent needs the exact text again, it pages it back through the handle.
- Evidence-Preserving Reducer: build and test logs of 4 KiB or more are handed to a cheaper model, which writes a short receipt of the evidence that matters. Plain code then checks that receipt: its format, a hash of the original, the exit status, the size, and that every quote appears word for word in the log. If any check fails, or the log looks like it holds credentials, the agent gets the raw log instead. The cheap model only extracts. The main model still decides what to do.
The paper traces how one of these was found. Analysis of traces showed edits and tests sitting next to each other often enough to project an 11.5% token saving. Telling the agent in its prompt to combine them did not work reliably, so the research agent changed the tool's definition to offer the combined action directly. It also invented its own progress measure, how often the fused action actually fired, and tracked it alongside the task score.
How much does it save, and compared with what?
The headline numbers come from the 51 public EdgeBench tasks, at API prices as of 17 August 2026. On GPT-5.6 Sol, the full SoL-Pi stack cost USD 894, against USD 1,339 for Pi and USD 1,787 for Codex. On Opus 5, it cost USD 1,158, against USD 1,741 for Pi and USD 2,535 for Claude Code.
Which comparison you quote changes the story. Against the stock harnesses, the cut is 50.0% versus Codex and 54.3% versus Claude Code. Against Pi, the harness SoL-Pi was built on, the cut is 33.2% and 33.5%. Pi already cost less than Codex and Claude Code in the paper's runs, so half or more of the gap was there before the search began (50% of it against Codex, 58% against Claude Code). SoL-Pi does not modify Claude Code or Codex. It is an extension for Pi.
The cost also falls by less than the token count, and the paper shows why. With GPT-5.6 Sol, the full stack halved the cached input it resent (from 2.13 billion to 1.06 billion tokens) but more than doubled the tokens written to the cache (from 0.014 billion to 0.032 billion), because compaction and excerpts change text the provider had already cached. Total cost still fell from USD 1,339 to USD 894. Keeping a long cached conversation intact is not automatically the cheapest choice.
The transfer result is the part to watch. The harness was searched only on GPT-5.6 Sol and then applied to Opus 5 without changes. The mechanisms fired less often on Opus 5, but the savings held at a similar level.
What does it cost in quality?
SoL-Pi is cheaper, not smarter. The full stack kept 93.7% of Pi's EdgeBench score on GPT-5.6 Sol (42.0 against 44.8) and 94.3% on Opus 5 (42.2 against 44.8). On 63 tasks from Terminal-Bench 4, it solved 15, while Codex and Pi each solved 18, though its cost per solved task was the lowest at USD 14.07. On six IMO 2026 problems that had to be proved in Lean 4, it proved three, as Pi did, while Codex proved five. Its cost per proved problem was again the lowest.
Single mechanisms behave differently from the full stack. Adding only ObservationPack to Pi raised the GPT-5.6 Sol score from 44.8 to 47.2, and adding only Action Fusion raised the Opus 5 score from 44.8 to 50.5, both while spending less than Pi. The full stack trades a few points of score for the deepest cut in cost.
The most interesting result is a team of agents. Given two hours to make a kernel run in fewer simulated machine cycles, a coordinator with 20 SoL-Pi workers reached 1,127 cycles for USD 60.11. The same setup with 20 Pi workers reached 1,366 cycles for USD 82.12, and a single Codex agent reached 1,333 cycles for USD 39.20. The cheaper workers turned the same two hours into the best result, though one agent alone was still the cheapest run.
How can you use it?
The code is public under the MIT licence at github.com/NVlabs/SoL-Pi. It installs on top of an unmodified Pi release, and every mechanism is off until you turn it on. The README also describes an example skill that hands a task from Claude Code or Codex to SoL-Pi in the background and reports the result back.
You can also copy the ideas without the code. The paper's add-one tests point to a simple habit: add one change at a time and measure it.
- Big outputs: send long command output to a file and give the agent the last 40 lines and the path, not the whole thing. Our own agent rules already ask for this by hand. ObservationPack does it automatically.
- Logs: let a cheap or local model summarise long logs, but accept the summary only if plain code confirms every quoted line exists in the original. Otherwise pass the raw log.
- Compaction: compact when a piece of work is finished, not in the middle of it, and only when the conversation is long enough that the savings beat the cache cost.
- Edits and checks: if your tools allow it, run the check in the same call as the edit.
- Your own loop: if you try an autoresearch loop on your setup, write the pass rules before you start, keep them out of the optimizer's reach, and test on tasks it never saw.
We have not run SoL-Pi on our own agents yet. The ideas map closely onto problems we already work around, and the first one we would try is the log reducer, because it gives a cheap local model a useful job while a deterministic check keeps it honest.
The takeaway: the same model got about a third cheaper on new tasks through four ordinary engineering changes, and an AI found them by reading another agent's traces under rules it could not bend. The paper is at arxiv.org/abs/2609.20519. We write about agents and the systems around them at https://myargoquest.com/blog and on https://x.com/myargoquest.