RRSI: Regularized Recursive Self-Improvement of Agent Harnesses | Summary

Contents

Summary
1. Introduction
2. Preliminaries
3. RRSI
4. Experiments
5. Related Work
6. Conclusion
Limitations
Appendix
Brief Thoughts

This article explains the key points of RRSI: Regularized Recursive Self-Improvement of Agent Harnesses.

2026-09-21 (arXiv)
Xia, Peng, Han, Rujun, Wang, Zifeng, Chen, Yanfei, Zhuang, Yufan, Lee, Yoonho, Huang, Chengsong, Yu, Han, CuiZhu, Zhongying, Ming, Yifei, et al.
Google Cloud AI Research, UNC-Chapel Hill, Stanford University, Washington University in St. Louis
Paper
Github
Project Page

Summary

Automatically improving an agent’s harness—the prompts, control flow, tools, memory, and context management surrounding a fixed language model—can overfit the tasks used to select changes. RRSI, Regularized Recursive Self-Improvement of Agent Harnesses, improves all six held-out evaluation splits relative to the initial harness in the main experiment; in the workspace setting, it consumes 2.42 million policy tokens, tokens used by the fixed language model, per trial versus 3.80 million for unregularized evolution.
RRSI regularizes candidate generation and selection while keeping every harness component editable. It gradually reduces the number of edits bundled into a candidate, retains evidence from previous evaluations, explores underused components, screens benchmark-specific code, and checks performance and inference cost before retaining changes.
The main evaluation spans eight benchmarks in coding, professional workspace tasks, and engineering design, with model weights frozen throughout search. After evolution on one suite per domain, unchanged harnesses improve SWE-bench Verified, a repository bug-fixing benchmark, by 1.8 points; three unseen workspace benchmarks by 3.5–4.7 points; and Frontier-Eng, an engineering optimization benchmark, by 4.3 Medal points, a score based on reaching ranked performance thresholds.
Removing either proposal or acceptance regularization reduces transfer despite increasing the score on the evolution tasks. Task-level uncertainty estimates and a second independent search support the reported gains, but the experiments cover bounded searches with fixed model weights and do not establish behavior during indefinite self-improvement or weight updates.

1. Introduction

Recursive self-improvement (RSI), here meaning repeated use of an agent system’s feedback to modify its own harness, reuses a finite evolve set: the tasks on which candidate changes are evaluated. This adaptive reuse can reward benchmark-specific patterns, favorable evaluation noise, and added computation without improving performance on unseen tasks.

The paper defines generalization as transferring an evolved harness to benchmarks with different task descriptions, tool interfaces, or verifiers, the programs that score task outcomes.
RRSI constrains how candidates are proposed and selected while allowing all harness components to change.
Figure 1 separates gains on the evolve set from gains on unseen workspace benchmarks; several prior methods improve the former while losing performance on the latter.

Evolution-set transfer and held-out performance across coding, workspace, and engineering design.

2. Preliminaries

The agent consists of a fixed backbone policy, the language model that chooses actions, and an editable harness. A task produces a trajectory, the sequence of the agent’s actions and observations, and a deliverable; a verifier assigns a score between 0 and 1. Performance is the expected verifier score, and cost is the expected number of policy tokens.

A verifier can be a coding test suite or a language model judging a workspace deliverable.
Ordinary harness evolution executes the current harness, summarizes its trajectories into feedback, proposes candidates, evaluates them on the same evolve set, and selects the highest-scoring candidate or retains the current harness.
Scores and costs are estimated by averaging k trials per task. Later proposals depend on earlier measurements of the same tasks, making the optimization adaptive.

3. RRSI

RRSI applies regularization, constraints intended to reduce overfitting, to iterative search over executable harness changes. Sparse updates limit how many mechanisms change together, historical evidence discourages repeated unsuccessful hypotheses, and conservative selection limits the retention of benchmark-specific content, evaluation noise, and unjustified resource growth.

3.1. A Regularization View of Harness Evolution

The reachable harness space remains open: prompts, control flow, configuration, context management, tools, skills, memory, and subagents can be modified, added, or removed. Proposal constraints control the number and types of edits explored; selection constraints determine which candidates can replace the current harness.

The L0-style analogy refers to bounding the number of independently attributable edits in one update.
Lasso/L1-style pruning refers to deleting unproductive discrete components. Ridge/L2-style acceptance refers to limiting aggregate resource growth without requiring a particular component to be removed.
These are functional analogies to classical regularization. RRSI does not optimize the corresponding norm-penalized objectives over a shared continuous parameter vector.

Proposal and selection regularization in the RRSI evolution loop.

3.2. Regularizing the Proposal Distribution

Candidate generation uses an annealed edit budget, a limit that decreases over search rounds; evidence-aware credit assignment, a record of earlier edits and their outcomes; and structured exploration of underused components. A cosine schedule lowers the maximum number of edits bundled into a candidate toward one, permitting coordinated early changes and making later results easier to attribute.

For every evaluated candidate, the history records the modified component, tested hypothesis, source diff, score and cost changes, and acceptance outcome. The proposer uses this history in later rounds, including unsuccessful results.
Search is considered stalled when progress over the previous w rounds does not exceed the empirical noise tolerance δ, estimated variation in repeated evaluations. During a stall, reserved proposal slots target component types that have not yet received a measured edit.
The edit pool can change across rounds. The budget constrains each update rather than permanently excluding component types.

3.3. Regularizing Candidate Selection

Candidate selection applies separate requirements for leakage, stability, and cost. Leakage means encoding content specific to the evolve benchmark in the harness: before full evaluation, a critic reads the source diff and screens task names, entity names, task-specific values, answers, other benchmark-specific logic, and inert machinery.

Repeated evaluations of the unchanged base harness establish δ. A candidate must score at least the running best accepted score minus δ, preventing successive small regressions from progressively lowering the reference score.
When a candidate’s score gain over the current harness exceeds δ, its relative token-cost increase must stay below a base allowance plus a gain-dependent allowance. Gains that do not exceed δ use a separate rule combining score change, cost change, and previously unused structural component types.
Components without a strictly positive measured contribution over the pruning window become deletion targets for subsequent proposals. Candidates must also pass any domain-specific checks; the selector then chooses the highest-scoring admissible candidate or retains the current harness if none qualifies.

4. Experiments

The experiments test comparative performance, transfer to unseen tasks and benchmarks, the contribution of the regularizers, and inference cost. The eight benchmarks comprise Terminal-Bench 2.1, terminal tasks, and SWE-bench Verified for coding; Harvey LAB, legal document work, and JobBench, GDPval, and APEX-Agents, professional workspace tasks; and EngDesign and Frontier-Eng for engineering design.

4.1. Experimental Setup

Evolution uses 89 containerized Terminal-Bench 2.1 tasks, a fixed 120-task Harvey LAB evolve set, or 61 EngDesign tasks. Harvey LAB also supplies a separate 40-task in-distribution held-out set, unseen tasks from the same suite; five other benchmarks provide out-of-distribution (OOD) evaluation, testing outside the suite used for search.

The main experiments freeze Claude Opus 4.8 as the backbone policy. The proposer, cross-round failure analyst, and leakage critic also use Claude Opus 4.8.
Coding starts from Terminus-2. Workspace and engineering settings use a ReAct loop, which combines reasoning and acting, over an MCP tool gateway exposing external tools, a dynamic toolbelt, and ReSum-style context management for controlling the model’s working context.
The four evolution baselines share the initial harness, frozen policy, evolve set, and candidate budget. They are Meta-Harness, executable harness-code optimization; Agentic Harness Engineering (AHE), evolution driven by execution observations; Test-Time Harness Evolution (TTHE), trace-based adaptation with fixed model weights; and HarnessX, adaptation over modular harness components. For Meta-Harness, see Meta-Harness: End-to-End Optimization of Model Harnesses | Summary.

4.2. Main Results

Against the initial harness evaluated in the same window, RRSI raises Terminal-Bench 2.1 from 74.2 to 80.2, Harvey LAB evolve from 89.4 to 90.5, and EngDesign from 50.0 to 54.9. All six held-out splits improve: SWE-bench Verified reaches 83.8 from 82.0, Harvey LAB held-out reaches 89.2 from 86.9, JobBench reaches 40.7 from 36.0, GDPval reaches 52.3 from 48.8, APEX-Agents reaches 37.9 from 34.2, and Frontier-Eng reaches 22.0 from 17.7.

In the workspace comparison, RRSI ties Meta-Harness on Harvey LAB held-out at 89.2 and exceeds every prior method on each of the three OOD benchmarks. Its OOD average is 43.6 versus 39.7 initially and approximately 40.6 for Meta-Harness.
Meta-Harness achieves the largest Harvey LAB evolve score, 93.0, compared with RRSI’s 90.5. The reversal between evolve-set and OOD rankings shows why evolution scores alone are insufficient to assess transfer.
Engineering transfer is scored by deterministic simulators or testbenches, extending the evidence beyond language-model judging. The abstract’s stated maximum OOD gain of 4.3 points conflicts with the 4.7-point JobBench gain in Figure 3 and Table 1.

Initial and evolved harness scores across eight benchmarks and nine evaluation splits.

Workspace benchmark comparison under a shared starting harness, policy, and candidate budget.

4.3. Analysis

The workspace ablation, a comparison that removes selected parts of the method, supports using proposal and acceptance regularization together. RRSI obtains an OOD average of 43.6 at 2.42 million tokens per trial; removing proposal regularizers yields 41.9 at 2.69 million, removing acceptance regularizers yields 41.0 at 3.59 million, and removing both yields 40.3 at 3.80 million.

Every ablated search scores higher than RRSI on the evolve set: 90.7 without proposal regularization, 91.5 without acceptance regularization, and 92.8 without either, compared with 90.5 for RRSI.
These grouped ablations assess the two sets of controls but do not isolate the contribution of each individual mechanism.
The tabulated token costs imply about 36% lower consumption than unregularized evolution, whereas the abstract reports 30%. RRSI still consumes more than the initial harness, which uses 1.56 million tokens per trial.

Grouped regularization ablations with evolution scores, held-out scores, and policy-token costs.

Independent coding searches with two backbone families both transfer to SWE-bench Verified. With Claude Opus 4.8, Terminal-Bench 2.1 improves by 6.0 points and SWE-bench Verified by 1.8; with Gemini 3.5 Flash, the corresponding gains are 14.1 and 2.2 points.

The Gemini 3.5 Flash search raises Terminal-Bench 2.1 from 64.6 to 78.7 and SWE-bench Verified from 76.8 to 79.0.
Running that final harness unchanged with the previously unused Gemini 3.1 Flash Lite raises Terminal-Bench 2.1 from 11.2 to 14.6, a 3.4-point absolute gain and 30.4% relative gain.
Independent searches across policy families and reuse with an unseen backbone are distinct tests. The latter is demonstrated on one benchmark with one additional backbone.

Independent coding evolution with two frozen backbone policies.

Reuse of a searched coding harness with an unseen evaluation backbone.

RRSI’s final workspace harness combines higher OOD performance with fewer policy tokens and shorter trajectories than the four prior evolved harnesses. It averages 26.3 steps per trial, compared with 27.3–34.6 for the prior methods and 21.2 for the initial harness.

AHE uses 3.82 million tokens per trial, reported as 58% more than RRSI, while scoring 4.4 points lower on the OOD average.
These measurements describe execution of the final harness. They do not establish the total cost of evolution, including proposal, criticism, and candidate evaluation.

Final workspace harness token cost, OOD performance, and trajectory length.

The paper connects harness engineering, automated harness evolution, and self-improving agents that search over their own code. Prior methods optimize entire harnesses or components such as skills and memory; concurrent work addresses generalization through explicit search objectives or archives that preserve diverse candidates.

RRSI’s stated distinction is its control of search dynamics: using the full edit history, filtering task-specific logic before scoring, and selecting candidates against a noise-adjusted performance reference.
These controls operate while the harness edit space remains open, separating the search procedure from the choice of editable components.

6. Conclusion

The conclusion argues for regularizing how repeated feedback from a finite evolve set becomes persistent harness changes. Across the tested domains, proposal and selection constraints improve held-out and cross-benchmark performance at lower final-harness execution cost than unregularized evolution.

Limitations

The study covers harness changes around frozen backbone models and does not evaluate evolution that updates model weights. Effectiveness may depend on feedback quality, regularization settings, and search budget; substantially different architectures, tool ecosystems, and longer self-improvement processes remain unvalidated.

Appendix

A. Limitations reiterates the restriction to frozen model weights and the dependence on feedback and search settings. B. Evaluation specifies matching evaluation windows, tool environments, judges, and trial counts for each harness and its baseline. B.1. Terminal-Bench 2.1 counts success only after hidden tests pass and rebuilds containers between comparison arms; B.2. SWE-bench Verified requires previously failing tests to pass and previously passing tests to remain passing.
B.3. Harvey LAB scores independently judged criteria, with 20–100 criteria per task and roughly 14,000 verdicts per full evaluation; missing deliverables fail their required criteria. B.4. JobBench uses its weighted rubric and averages judgments from Gemini-3.5-Flash and Claude Opus 4.8. B.5. GDPval compares outputs with human expert deliverables using three judges and both presentation orders; the text reports 185 tasks and 204 comparisons per judge without reconciling those counts.
B.6. APEX-Agents evaluates all 480 tasks using pass@1, success on a single sampled rollout, and counts missing rollouts as failures. B.7. EngDesign evolves on all 61 license-free tasks without an in-distribution held-out split. B.8. Frontier-Eng reports Medal Score, mean credit for reaching gold, silver, or bronze thresholds worth 1, 0.67, or 0.33; overlapping EngDesign tasks are excluded, and unavailable environments earn no credit, leaving 38 of 47 tasks contributing credit under the reported shared denominator.
C. Baseline Methods describes Meta-Harness as executable-code optimization using accumulated traces, AHE as evolution driven by execution observations, TTHE as trace-based test-time adaptation, and HarnessX as adaptation over modular typed components. D. Method Details emphasizes that the classical regularization terms describe analogous functions rather than norm-penalized optimization. D.1. Round-Level Formulation specifies candidate generation from current feedback, edit history, edit budget, exploration directives, and pruning targets, followed by selection among admissible candidates or retention of the current harness.
D.2. Proposal-Side Bookkeeping records each atomic edit’s component, hypothesis, source diff, measured score and cost changes, and acceptance flag; bundled edits share their candidate’s measurements. D.3. Selection-Side Bookkeeping awards novelty credit only to client_tool, skill, memory, and subagent, and combines score change, relative cost change, and structural novelty when gains do not exceed the noise tolerance. Coding assigns zero weight to within-band score gains, whereas workspace and engineering settings assign positive weight; the text refers readers to Table 5 for these weights, but the table omits them.
E. Experiments and E.1. Hyperparameter Setting report 20, 20, and 40 evolution rounds and 2, 2, and 4 trials per task for coding, workspace, and engineering, respectively; the noise tolerances are 0.017, 0.004, and 0.020, initial edit budgets are 4, 3, and 4, and the final budget is 1 throughout. Settings are fixed before each run without consulting held-out or OOD benchmarks; noise tolerance comes from base-harness evaluations, and cost allowances are calibrated against opening-round results. E.2. Reliability of the Reported Gains uses 10,000 paired task-level bootstrap resamples, repeatedly sampling matched tasks with replacement, to obtain positive 95% confidence intervals for every tabulated gain; a second independent search also improves every split, with workspace OOD averages of 43.6 and 43.7.
F. Qualitative Case Study illustrates the selection rules: coding R0-A is accepted with a 3.93-point gain, R0-B is rejected with a 1.69-point gain and 26.1% higher cost, and R8-B is rejected after losing 2.81 points despite saving 13.6% in cost. An engineering recovery hint is accepted after passes rise from 122/244 to 128/244 with 1.6% more tokens; the final workspace harness retains four edits from 32 evaluated candidates over 20 rounds, comprising a coverage recheck and three deliverable-inspection probes, which the authors contrast with scorer-oriented or document-genre-specific instructions retained by Meta-Harness and TTHE. Code and trajectories are linked at https://github.com/google-research/rrsi and https://regularized-rsi.com/.

Brief Thoughts

The clearest result is the reversal between evolution scores and transfer: RRSI has a smaller evolve-set gain while outperforming unregularized search and prior methods on unseen workspace benchmarks. Deterministic engineering evaluation, positive task-level confidence intervals, and a second search strengthen this finding, although grouped ablations leave the contribution of individual mechanisms unresolved. The missing within-band weights and unreconciled GDPval counts are reproducibility gaps in the PDF. Two searches per main setting provide limited evidence about search variability. The unseen-backbone transfer test adds evidence of portability on one benchmark, without establishing transfer across arbitrary backbones. The efficiency advantage concerns execution relative to other evolved harnesses. RRSI remains more expensive than the starting harness, and the final-harness token measurements do not establish total search cost.