Every time one of our agents fails a task in a new way, a person opens a file and adds a rule. "Reopen the file after you write it." "Don't search the web for facts already in the workspace." "Run the tests before you say done." The rule goes into the agent's instructions, and the next run is a little better. Over a year this is how an agent setup actually gets good: not through a better model, but through a few hundred small edits to the scaffolding around it.
The problem is that the person is the bottleneck. They see a failure, guess a fix, type it in, and never run the old and new version side by side to check the guess. Nobody keeps a record of which fixes worked. And the fixes that help on one kind of task rarely get carried to the next.
On 2026-10-09, researchers at Google Cloud AI Research published a paper called Harness²: Recursive Agent Harnessing for an Open World (https://arxiv.org/abs/2610.14224). It describes a system in which the agent does this editing work itself, at the moment a new task arrives, with no answer key to check against, and then learns from each round of editing so the next round is better. This post explains how it works, what the numbers say, where it breaks, how to build a small version of it, and what we are changing in our own setup because of it.
What is a harness?
A harness is everything around the model that is not the model: the system prompt, the rules it must not break, its memory file, the skills it can load, the tools it can call and the guidance on using them, the helper scripts it can run, the sub-agents it can delegate to, and any hooks (small programs that run around each tool call and can stop it) that enforce behaviour instead of asking for it. The paper's phrase is "Agent = Model + Harness", and its starting point is that an agent's capability depends on the harness as well as the model. In its experiments, a weaker model with an improved harness scored higher than a stronger model with its stock one.
The paper splits the harness into eight files. A system prompt, a guardrails file, a memory file and a tool-guidance file that the model sees on every call. A skills folder and a scripts folder that cost nothing until invoked. A sub-agents folder for work that needs its own context window. And a plugin folder for code that runs inside the tool loop, "outside the model's discretion". Because each is a file, a harness change is a diff, and a diff is something another agent can write.
An analogy: the model is a new employee with good general ability, and the harness is the onboarding binder, the checklists, the templates and the locked cabinets. Changing the binder is cheaper than hiring someone smarter. The analogy stops working at one point, which is the whole subject of the paper: here the binder rewrites itself.
Why is editing without a grader hard?
Earlier work on automatic harness improvement used a training set with known answers. Try a change, score it, keep it if the score goes up. That works until deployment, where the tasks are new and there is no score. The paper calls this the open-world setting: tasks "are typically unseen during the harness design phase and novel at test time, leaving no ground-truth verifiers".
The obvious shortcut is to ask a model whether the new output looks better. The paper is careful not to do this. Model judgments and execution traces "can guide revision without establishing correctness", and the authors reserve the benchmark scores strictly for evaluation. Their editing sessions are not even allowed to see a rubric: a guard "refuses to hand over any working directory that contains a grader verdict, rubric, or score".
So the question becomes: what evidence can you trust when you have no answer key?
How does one improvement step work?
The paper's answer is a three-part step it calls Propose, Probe, Compose. It works on one task.
Propose. First the agent runs the task once with the current harness. This is the reference run, and the paper says what it is for: "The reference reveals how the harness behaves, not whether its deliverable is correct." Then two separate editing sessions each read the task, the harness and the full trajectory, and each proposes an edited harness. One session owns how the agent works: planning, tool use, context management, finishing. The other owns the domain: the knowledge, procedures and constraints of this kind of task. The editing prompt's central rule is one sentence: "Every edit must trace to a moment in a trajectory."
Probe. Each edited harness is run on the same task, with the same model and the same environment. Now there are three trajectories. Comparing each candidate with the reference shows whether the edit removed the failure it targeted, changed nothing, or broke something else. The paper calls this a behavioral contrast, and insists it is evidence about behaviour, not proof of correctness.
Compose. A third session reads the reference, both candidates, their diffs and all three runs, and builds the final harness. It may merge the supported edits, fix a conflict between them, adopt one candidate whole, or keep the original. "Each decision is tied to a concrete observation rather than a presumed score gain." The composed harness runs once more, and that run's deliverable is what the task returns.
The step can be widened or deepened. With a budget of k, the parallel form runs k reference runs, proposes and probes k pairs of candidates, and runs k composers over the whole pool. The sequential form chains k steps, each seeing everything before it. When several deliverables exist, a selector picks one. The selector is deliberately blind: it compares the anonymised outputs against the task request "without access to its originating harness or synthesizing new content", and never sees a score.
The control condition is the part to remember. The authors also tried letting an editor rewrite the harness directly, from the task alone, with no reference run and no probing. That made things worse than not editing at all: the Code score fell from 73.4 to 63.2. Edits that never ran are guesses, and guesses typed into an instruction file are a tax on every later run.
How does the system get better at improving?
This is the second recursion, and the one most self-improvement setups skip. Each task starts from the same base harness, so nothing task-specific leaks forward. What does carry forward is the experience of improving.
After a task finishes, a reflector session reviews the whole record: the reference run, both candidates, the probes, the composer's decisions and the selection. It writes two things into a library: the adopted harness itself, executable files and all, and a short EXPERIENCE.md that records the situation, the recurring weakness, the lesson, and the evidence. The rule for what gets kept is strict: "Lessons are retained only when supported by execution contrasts; selecting an output alone does not establish transferability." If the evidence is thin, nothing is written.
The next task's editors read the library's index before proposing. They may copy a skill or script from an earlier entry, but the prompt warns them that "an earlier task's mapping or threshold is a hypothesis here, not a fact" and must be re-verified against the current trajectory.
The paper measured what this is worth. Removing the library cost about three points on one benchmark. A library built while solving one benchmark still helped on a different one. And the more exploration a task did, the more useful the library it left behind: a library built at a budget of five candidate pairs per task kept helping later tasks even when those tasks did no probing at all.
What do the numbers say?
The paper tests on three professional benchmarks: WorkBuddy-Bench (code, web and office deliverables), JobBench (long professional tasks judged against a rubric) and Harvey's Legal Agent Benchmark. The models being improved are Gemini 3.5 Flash and GLM-5, each frozen inside two off-the-shelf coding-agent harnesses, OpenCode and Codex. The editor is Claude Opus 4.8. Each number is the mean of three runs with the tasks in a different order.
- One step helps everywhere. A single Propose, Probe, Compose step improved every model and harness combination on every benchmark. The largest one-step gain was 11.5 points for GLM-5 on WorkBuddy-Bench and 11.8 percentage points on the legal benchmark.
- More budget, more gain. Scores rose with every step of budget from one to five. At a budget of five, parallel recursion added a further 2.9 to 3.9 points on WorkBuddy-Bench and up to 6.7 on the legal benchmark. On the two domains plotted, both recursive forms at a budget of two already beat every plotted baseline at five.
- The editor does not have to be a stronger model. Using the same model as editor and executor, all five models tested improved, including Claude Opus 5, which gained between 3.0 and 5.5 points depending on the benchmark.
- The value lives in skills and scripts, not prompt text. When the authors removed parts of a finished harness, dropping the skills cost 5.3 points on average, dropping the scripts 3.7, and dropping the edited prompt text only 1.5. About half of all edits the editors made were skills and scripts.
- A memory of past strategies can hurt. The ReasoningBank baseline, which keeps the harness fixed and prepends distilled lessons from similar past tasks, fell as much as 6.6 points below the unmodified harness on coding tasks.
- It costs real compute. Every step adds two candidate runs, a composed run and several editing sessions. The paper's compute plot measures total spend per task in millions of tokens, and the authors are plain about the cost in their limitations section.
Where does it break?
The appendix lists nine cases where the improved harness scored below the reference, and they are more instructive than the wins.
The blind selector sometimes picks the wrong output. On one coding task it chose an output that scored 55 over one that scored 95, describing it as "the complete, plainest implementation". On a legal task it chose the most precise answer when the criteria rewarded coverage.
A wrong edit can poison every candidate. In one case the editor wrote a rule saying a certain behaviour "must not be 'fixed'", and the rule was wrong; every candidate inherited it and all of them scored lower. In another, the editor wrote its own checker script, the script reported PASS three times, and the bug was in the editor's own assumption. The paper's sentence on this one is worth keeping: "A checker written by the editor cannot catch the editor's assumption."
And sometimes an edit simply buys nothing. One task gained a correct PDF-filling skill with a verifying script, which consumed 85 of 132 steps, and the grading criteria never looked at those PDFs.
How would you build a small version?
Everything in the paper is files and sessions, so a minimal version needs no framework. The authors' code is on GitHub at https://github.com/google-research/harness2 under the Apache 2.0 licence if you want the full pipeline. The shape of a small one:
- Make the harness files. Eight files or folders, with a one-line placeholder in each empty one. The placeholder is what lets an editing session discover that the lever exists.
- Record every run. Each run writes its full tool-call trajectory and its deliverable to disk, as files an editor can read. Nothing about the step works without this.
- Reference run. Run the task once with the current harness.
- Two scoped editors. One prompt owns how the agent works, the other owns the domain. Each must point every edit at a line in the trajectory and choose the cheapest component that fixes it: a script for mechanical repetition, a skill for a triggered multi-step procedure, always-on prose only for ordinary sequencing, and a hook only when there is repeated evidence that clear prose is being ignored.
- Probe. Run each edited harness on the same task.
- Compose. A third session keeps only edits with a visible difference between runs, then runs the result.
- Select blind. If there is more than one deliverable, a session that cannot see the harnesses or any score picks one.
- Reflect and store. Write the adopted harness and an experience note (situation, lesson, evidence) into a library with an index. Skip the entry when the evidence is weak.
- Keep the grader out. Whatever scores the work, whether tests, a rubric or a person, never enters an editing session.
- Reset. The next task starts from the base harness, with only the library changed.
A prompt fragment the editors in the paper use, in plain words: trace the workflow, find the earliest observed divergence, match it to the cheapest effective component, never prescribe blind repetition, and remember that "a successful write call does not establish usability", so the deliverable must be reopened or mechanically checked.
What's built and what's next
We run a company of AI agents on a shared task board, and our harness already has the paper's eight parts: shared instruction files, a guard hook that blocks a short list of dangerous commands, a finish hook that demands a dev-log entry, shared skills, verifier scripts that a dispatcher runs after a card completes, a read-only fact-checking sub-agent, and memory files. We also already split instructions the way the paper splits editors: an engine file for how agents work, and local files for the domain.
What we do not have is either recursion. Every one of our harness rules was written by a person after an incident, and none was tested against the version before it. Our retries tell the agent to "make it pass" with the failing output attached, a single candidate with no contrast. And we keep no record of which fixes worked, so lessons live in commit messages.
Because of this paper we are making four changes, in this order. First, an experience library: one note per adopted harness change, citing the run that showed the problem and the run that showed the fix worked, with an index that any session proposing a harness change must read first. Second, an evidence rule: our finish hook will refuse a change to the instruction files, skills or hooks that does not come with such an entry. Third, probing on failed cards only: when a verifier fails, the dispatcher will open two scoped retries, one for procedure and one for domain, run the verifier on both, and let a harness-blind selector choose if both pass. Fourth, moving procedures out of the always-on instruction text into skills with scripts, since that is where the paper found the value.
Two things we are deliberately not doing. We will not let an agent edit the guard that blocks irreversible actions; the paper's own failure table shows what a wrong rule does when every candidate inherits it. And we are auditing our memory files to keep only facts, since a memory of strategies was the baseline that fell furthest below doing nothing.
The takeaway is smaller than "agents that improve themselves". It is that an instruction you add without running it is a guess, and a system that records which guesses were right is the one that gets better. The paper is at https://arxiv.org/abs/2610.14224. We post what we build at https://myargoquest.com/blog and https://x.com/myargoquest.