Literature
Display
← All papers

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Peng Xia · Rujun Han · Zifeng Wang · Yanfei Chen · Yufan Zhang · Yoonho Lee · Chengsong Huang · Han Yu · Zhongying CuiZhu · Yifei Ming · Huaxiu Yao · Burak Gokturk · Tomas Pfister · Chen-Yu Lee

· arXiv (preprint) ↗

AI agents10 min text · CondensedOriginal paper ↗
Contents

At a glance

RRSI improves an agent’s surrounding code, prompts, tools, memory, and control flow while keeping model weights fixed. It limits edit bundles, remembers previous experiments, screens benchmark-specific changes, and accepts candidates using performance, noise, and token-cost criteria. Across eight benchmarks, the reported harnesses improve all six held-out splits. The largest 14.1-point gain occurs on the coding evolution benchmark; unseen-benchmark gains reach 4.7 points. The final workspace harness uses fewer tokens than unregularized evolution, but more than the initial harness. Code and trajectories are available.

GitHub repositoryProject website

1 Introduction

Repeatedly proposing and scoring harness changes on the same finite task set can fit benchmark details, reward evaluation noise, and accumulate costly machinery. RRSI addresses this adaptive overfitting by regulating how candidates are proposed and retained. A separate harness is evolved for each domain and then transferred unchanged to that domain’s unseen benchmarks; the paper does not demonstrate one universal harness across all domains. [23] [47] [5]

Original figure
Figure 1: Evolution overfits on the split it is scored on whereas RRSI generalizes the improvements. (a) Gains on the evolve split against gains out of distribution for the agentic workspace benchmark. Prior methods retain little of their evolve-set gain and several end below H0H_{0}, the initial harness. (b-d) Out-of-distribution held-out score for H0H_{0}, average of the four baseline methods and our RRSI on SWE-bench Verified, the mean of JobBench, GDPval and APEX-Agents, and Frontier-Eng.

2 Preliminaries

An agent combines a frozen policy with an editable harness. The harness controls prompts, tools, memory, context, and when the policy acts or stops. Evolution repeatedly executes the agent on an “evolve” set, proposes changes from its traces, and measures candidate scores and policy-token use. Reusing these tasks makes the search adaptively dependent on earlier measurements, so its best observed score is insufficient evidence of generalization.

Equations 1–3
S(H;𝒟)=𝔼x∼𝒟𝔼τ∼A(⋅∣x)[r(x,τ)],C(H;𝒟)=𝔼x∼𝒟𝔼τ∼A(⋅∣x)[c(τ)],S(H;\mathcal{D})=\mathbb{E}_{x\sim\mathcal{D}}\,\mathbb{E}_{\tau\sim A(\cdot\mid x)}[r(x,\tau)],\qquad C(H;\mathcal{D})=\mathbb{E}_{x\sim\mathcal{D}}\,\mathbb{E}_{\tau\sim A(\cdot\mid x)}[c(\tau)], (1)
ℋt={Ht(1),…,Ht(mt)}∼P0(⋅∣Ht,ℱt),Ht+1=arg​maxH′∈ℋt∪{Ht}S^(H′;𝒟evolve),\mathcal{H}_{t}=\{H_{t}^{(1)},\ldots,H_{t}^{(m_{t})}\}\sim P_{0}(\cdot\mid H_{t},\mathcal{F}_{t}),\qquad H_{t+1}=\operatorname*{arg\,max}_{H^{\prime}\in\mathcal{H}_{t}\cup\{H_{t}\}}\hat{S}(H^{\prime};\mathcal{D}_{\mathrm{evolve}}), (2)
S^​(H)=1k​|𝒟evolve|​∑x∈𝒟evolve∑j=1kr⁡(x,τx(j)),C^​(H)=1k​|𝒟evolve|​∑x∈𝒟evolve∑j=1kc⁡(τx(j)).\hat{S}(H)=\frac{1}{k|\mathcal{D}_{\mathrm{evolve}}|}\sum_{x\in\mathcal{D}_{\mathrm{evolve}}}\sum_{j=1}^{k}r(x,\tau_{x}^{(j)}),\qquad\hat{C}(H)=\frac{1}{k|\mathcal{D}_{\mathrm{evolve}}|}\sum_{x\in\mathcal{D}_{\mathrm{evolve}}}\sum_{j=1}^{k}c(\tau_{x}^{(j)}). (3)

3 RRSI

RRSI leaves every harness component editable. Proposal rules constrain the amount and direction of change; selection rules constrain what survives. The intent is to retain reusable mechanisms while resisting benchmark-specific shortcuts, noisy improvements, and unjustified computation.

Refer to caption
Figure 2: Overview of RRSI. RRSI regularizes the search trajectory, not restricting the potential harness edit space: proposal-side constraints control how search capacity is used, while selection-side constraints control which measured improvements are allowed to become a permanent state.

3.1 A Regularization View of Harness Evolution

The paper draws functional analogies to classical regularization: an edit-count limit resembles sparsity, pruning removes unproductive components, and cost-aware acceptance restrains aggregate resource growth. These are discrete search heuristics, not optimization of literal L0, L1, or L2 norm penalties. [13] [7] [31]

3.2 Regularizing the Proposal Distribution

Proposal regularization combines a shrinking edit budget, a history of tested hypotheses, and explicit exploration when recent progress stalls.

L 0 -Style Annealed Update Sparsity.

A cosine schedule reduces the number of independent edits permitted in each candidate from an initial maximum toward one. Early rounds can explore coordinated changes; later rounds make results easier to attribute to individual mechanisms.

Equation 4
bt=⌈bmin+(bmax−bmin)⋅12​(1+cos⁡(π​t/T))⌉.b_{t}=\Big\lceil b_{\min}+(b_{\max}-b_{\min})\cdot\tfrac{1}{2}\big(1+\cos(\pi t/T)\big)\Big\rceil. (4)

Evidence-Aware Credit Assignment.

Each evaluated edit records its component, hypothesis, code diff, score and cost changes, and acceptance outcome. Later proposals consult both positive and negative evidence, reducing repeated tests of already unsuccessful ideas. Bundled edits still share a candidate-level score, so this history does not provide causal attribution to every individual edit. [6]

Structured Exploration.

When improvement over a recent window stays within the estimated noise band, part of the proposal budget targets component types not yet tested. This encourages exploration beyond repeated prompt rewriting without forbidding any component.

3.3 Regularizing Candidate Selection

A candidate must pass all applicable screening, stability, cost, and domain checks before competing on measured score. A high score cannot compensate for failure of one of these checks.

Leakage Screening.

A critic rejects source edits containing benchmark-specific task names, entities, answers, or values, as well as inert additions, before full evaluation. Generic mechanisms remain admissible. This targets explicit leakage in proposed code; it is not proof against every form of benchmark adaptation or backbone contamination.

Stability-Aware Acceptance.

Repeated runs of the unchanged base harness estimate a noise tolerance. A candidate must remain within that tolerance of the best accepted score so far, preventing a chain of individually small regressions from drifting steadily downward. The tolerance is empirical, rather than a general statistical guarantee for the adaptive search.

Equation 5
S^​(H′)≥S⋆−δ.\hat{S}(H^{\prime})\geq S^{\star}-\delta. (5)

Ridge/ L 2 -Style Complexity-Aware Acceptance.

When a candidate’s measured gain exceeds the noise tolerance, its relative token-cost increase must stay below a gain-dependent allowance. Larger gains can justify more computation. Token use is the resource proxy; it does not measure every tool, simulator, or wall-clock cost.

Equations 6–7
Δ​S=S^​(H′)−S^​(Ht),Δ​C=C^​(H′)−C^​(Ht)C^​(Ht).\Delta S=\hat{S}(H^{\prime})-\hat{S}(H_{t}),\qquad\Delta C=\frac{\hat{C}(H^{\prime})-\hat{C}(H_{t})}{\hat{C}(H_{t})}. (6)
Δ​C≤β0+β1​Δ​S.\Delta C\leq\beta_{0}+\beta_{1}\Delta S. (7)

Lasso/ L 1 -Style Structural Pruning.

Components with no positive measured gain during a recent pruning window become proposed deletion targets. This sparsifies retained structure, whereas the annealed budget limits each update. Deletions still pass through candidate evaluation and acceptance.

4 Experiments

Evaluation spans coding, professional workspace tasks, and engineering design. It asks whether gains survive new tasks and benchmarks, which regularizers matter, and how much inference the evolved harness consumes.

4.1 Experimental Setup

The evolution sets are 89 Terminal-Bench tasks, 120 Harvey LAB tasks, and 61 license-free EngDesign tasks. Held-out evaluation uses 40 additional Harvey tasks plus SWE-bench Verified, JobBench, GDPval, APEX-Agents, and Frontier-Eng. Comparisons share initial harnesses, frozen policy, evolve tasks, and candidate budgets. Main runs use Claude Opus 4.8 for acting, proposing, analysis, and criticism; coding also tests Gemini 3.5 Flash. Workspace and engineering harnesses use ReAct, an MCP gateway, a dynamic toolbelt, and context summarization. [33] [12] [10]

4.2 Main Results

With the main backbone, evolve-set gains are 6.0 points on Terminal-Bench, 1.1 on Harvey LAB, and 4.9 on EngDesign. Held-out gains are 1.8 on SWE-bench, 2.3 on Harvey, 4.7 on JobBench, 3.5 on GDPval, 3.7 on APEX, and 4.3 Medal points on Frontier-Eng. On the three unseen workspace benchmarks, the mean rises from 39.7 to 43.6 despite a smaller evolve-set improvement than prior methods. RRSI ties Meta-Harness on held-out Harvey and exceeds the compared methods on the workspace OOD scores.

Engineering tasks use deterministic simulators rather than language-model judges, so those gains do not depend on a judge’s writing preferences. Deterministic grading nevertheless leaves policy sampling and finite benchmark coverage as sources of uncertainty.

Original figure
Figure 3: Main results in all three domains.
In-Distribution Out-of-Distribution
Method Harvey LAB (Evolve) Harvey LAB (ID Held-out) JobBench GDPval APEX-Agents
H0H_{0} (no evolution) 89.4 86.9 36.0 48.8 34.2
Meta-Harness (Lee et al., 2026b) 93.0 89.2 37.1 49.1 35.7
AHE (Lin et al., 2026a) 90.7 88.7 37.2 47.2 33.1
TTHE (Nie et al., 2026) 91.1 88.5 35.2 47.0 31.7
HarnessX (Chen et al., 2026) 91.8 89.1 36.3 48.5 34.3
RRSI (ours) 90.5 89.2 40.7 52.3 37.9
Table 1: Comparison with prior harness evolution methods on agentic workspace tasks.

4.3 Analysis

Removing proposal or acceptance regularization raises the evolve-set score while reducing workspace transfer. Removing both gives a 40.3 OOD average at 3.80 million policy tokens per trial, compared with 43.6 at 2.42 million for RRSI. The initial harness costs 1.56 million tokens: RRSI is cheaper than the evolved alternatives, while still spending more than the starting system.

Separate coding evolution with Gemini 3.5 Flash improves Terminal-Bench from 64.6 to 78.7 and unseen SWE-bench from 76.8 to 79.0. Running that evolved harness with the previously unused Gemini 3.1 Flash Lite raises Terminal-Bench from 11.2 to 14.6. This supports transfer across these backbones, rather than universal independence from model choice.

Variant Harvey LAB (Evolve) Harvey LAB (ID Held-out) OOD Avg. Tokens/trial (m) ↓\downarrow
H0H_{0} (no evolution) 89.4 86.9 39.7 1.56
Unregularized evolution 92.8 88.9 40.3 3.80
w/o proposal regularizers 90.7 88.8 41.9 2.69
w/o acceptance regularizers 91.5 88.7 41.0 3.59
RRSI 90.5 89.2 43.6 2.42
Table 2: Ablation study of the regularizers on agentic workspace tasks. OOD Avg. is the mean over JobBench, GDPval and APEX-Agents.
Policy Benchmark H0H_{0} RRSI Δ\Delta
Claude Opus 4.8 Terminal-Bench 2.1 (Evolve) 74.2 80.2 +6.0
SWE-bench Verified (OOD) 82.0 83.8 +1.8
Gemini 3.5 Flash Terminal-Bench 2.1 (Evolve) 64.6 78.7 +14.1
SWE-bench Verified (OOD) 76.8 79.0 +2.2
Table 3: Policy robustness in the coding domain. Harness evolution is run independently with each frozen policy on Terminal-Bench, and the harness is evaluated unchanged on SWE-bench Verified.
Evaluation policy H0H_{0} RRSI Δ\Delta
Gemini 3.5 Flash (search policy) 64.6 78.7 +14.1
Gemini 3.1 Flash Lite (unseen) 11.2 14.6 +3.4
Table 4: Cross-model transfer on Terminal-Bench 2.1. The harness evolved with Gemini 3.5 Flash as the frozen policy is run unchanged with a weaker backbone that never took part in the search.
Original figure
Figure 4: Cost of the final harness of each arm, measured on the evolve split of the agentic workspace instance. OOD Avg. is the mean over JobBench, GDPval and APEX-Agents. The shaded region in (a) is everything RRSI dominates: more policy tokens per trial for a lower out-of-distribution average.

5 Related Work

Prior work evolves complete harnesses or individual skills, memories, and preferences, generally retaining changes by benchmark score. Other concurrent approaches explicitly target generalization or preserve diverse candidates. RRSI contributes proposal and selection constraints that can operate across different editable components. [23] [25] [34] [3] [60] [32]

6 Conclusion

Automatically editing an agent’s scaffold can improve held-out performance when the search is constrained by evidence, uncertainty, and cost. The experiments support these particular short evolution runs; they do not establish unrestricted recursive improvement or changes to the underlying model’s learned weights.

Limitations

The study keeps backbone weights frozen and depends on finite evolve sets, feedback quality, search budgets, and several hyperparameters. Broader architectures, tool ecosystems, and longer runs remain untested. Judge-based workspace scores measure rubric or preference outcomes, while token comparisons concern running the resulting harness and do not establish lower total development cost. Appendix reporting also leaves some reproducibility details unresolved, noted below.

Appendix A Evaluation

Each evolved harness and its baseline are evaluated in the same time window with matching environments, judges, and trial counts. Grading differs by benchmark, so score units should not be treated as interchangeable.

A.1 Terminal-Bench 2.1

Terminal-Bench runs 89 isolated container tasks. Success requires passing hidden task tests after execution; containers are rebuilt between comparison arms. [33]

A.2 SWE-bench Verified

SWE-bench Verified requires a repository patch that fixes designated failing tests while preserving previously passing tests. The metric is the fraction of issues satisfying both requirements. [18]

A.3 Harvey LAB

Harvey LAB tasks require named deliverable files built from source documents. Gemini 3.5 Flash grades each rubric criterion independently; the aggregate weights tasks by their number of criteria, and missing files fail their criteria. The 120/40 evolve/held-out partition stays fixed. [12]

A.4 JobBench

JobBench provides professional task folders while withholding some reference material to require retrieval. The harness uses files, code, and web search; scores use weighted rubrics averaged across Gemini 3.5 Flash and Claude Opus 4.8 judges. [24]

A.5 GDPval

GDPval compares agent deliverables with supplied human-expert outputs using three judges from different model families. Both presentation orders are judged and majority preference determines the outcome. The appendix reports 185 tasks but also 204 comparisons per judge, an unresolved accounting inconsistency. Treat its reported win rate as the authors’ benchmark result, rather than inferring a trial count from those conflicting figures. [38]

A.6 APEX-Agents

APEX-Agents supplies professional sandbox worlds with MCP tools. A single rollout must satisfy the task rubric to count as a success. All 480 tasks remain in the denominator; infrastructure failures receive zero credit. [44]

A.7 EngDesign

EngDesign uses 61 tasks runnable without proprietary simulators. Task-specific deterministic checks grade physical designs; all tasks are used for evolution, leaving no internal holdout. Transfer is instead checked on Frontier-Eng. [10]

A.8 Frontier-Eng

Frontier-Eng awards 1, 0.67, or 0.33 credit against frozen gold, silver, and bronze feasible-result thresholds. The overlapping EngDesign domain is excluded. The paper retains a 47-task denominator, with only 38 tasks able to contribute credit after exclusions and unavailable environments; both arms use the same accounting. It serves only as a transfer benchmark. [4]

Appendix B Baseline Methods

Meta-Harness searches executable harness code using accumulated traces and scores. AHE organizes component observations and edit outcomes; TTHE adapts multiple executable candidates during test time; HarnessX composes typed prompts, tools, memory, and control-flow primitives. All four are compared from the same initial scaffold and candidate budget. [23] [25] [34] [3]

Appendix C Method Details

The appendix formalizes search over editable programs. The regularization analogies describe functional roles, without assuming a shared continuous parameter vector or norm-penalized objective.

C.1 Round-Level Formulation

Each round combines current feedback, edit history, the shrinking budget, exploration directives, and pruning targets to propose candidates. After screening and evaluation, the highest-scoring admissible candidate replaces the incumbent; if none qualifies, the current harness remains. The two complete algorithms and the round-level equations are retained.

Equation 8
ℋt∼Preg(⋅∣Ht,ℱt,ℒt,bt,ℰt,ℬt)⊆Ω(Ht),Ht+1=arg​maxH′∈ℋt∩𝒜tS^(H′),\mathcal{H}_{t}\sim P_{\mathrm{reg}}\big(\cdot\mid H_{t},\mathcal{F}_{t},\mathcal{L}_{t},b_{t},\mathcal{E}_{t},\mathcal{B}_{t}\big)\subseteq\Omega(H_{t}),\qquad H_{t+1}=\operatorname*{arg\,max}_{H^{\prime}\in\mathcal{H}_{t}\cap\mathcal{A}_{t}}\hat{S}(H^{\prime}), (8)
Algorithm 1 RRSI, proposal side.
1: HtH_{t}, history ℒt\mathcal{L}_{t}, round tt of TT
2: bmin,bmaxb_{\min},b_{\max}, stall window ww, noise band δ\delta
3: ℱt←Analyze​(Ht,𝒟evolve)\mathcal{F}_{t}\leftarrow\textsc{Analyze}(H_{t},\mathcal{D}_{\mathrm{evolve}})
4: bt←⌈bmin+(bmax−bmin)​12​(1+cos⁡π​tT)⌉b_{t}\leftarrow\big\lceil b_{\min}+(b_{\max}-b_{\min})\tfrac{1}{2}(1+\cos\tfrac{\pi t}{T})\big\rceil
5: ⊳\triangleright L0L_{0}-style edit-cardinality control
6: σt←𝟙[S^t−S^t−w≤δ]\sigma_{t}\leftarrow\mathbb{1}[\hat{S}_{t}-\hat{S}_{t-w}\leq\delta]
7: 𝒯t←{ℓi:(ti,ℓi,…)∈ℒt}\mathcal{T}_{t}\leftarrow\{\ell_{i}:(t_{i},\ell_{i},\ldots)\in\mathcal{L}_{t}\}
8: 𝒰t←𝒦∖𝒯t\mathcal{U}_{t}\leftarrow\mathcal{K}\setminus\mathcal{T}_{t}
9: ℰt←(σt,𝒰t,mdraft)\mathcal{E}_{t}\leftarrow(\sigma_{t},\mathcal{U}_{t},m_{\mathrm{draft}})
10: ℬt←{ℓ∈𝒯t:gt​(ℓ)≤0}\mathcal{B}_{t}\leftarrow\{\ell\in\mathcal{T}_{t}:g_{t}(\ell)\leq 0\}
11: ⊳\triangleright Lasso/L1L_{1}-style pruning targets
12: ℋt∼Preg(⋅∣Ht,ℱt,ℒt,bt,ℰt,ℬt)\mathcal{H}_{t}\sim P_{\mathrm{reg}}(\cdot\mid H_{t},\mathcal{F}_{t},\mathcal{L}_{t},b_{t},\mathcal{E}_{t},\mathcal{B}_{t})
13: tag each atomic edit with component and hypothesis metadata
14: return candidates that pass the pre-evaluation screen
Algorithm 2 RRSI, selection side.
1: screened ℋt\mathcal{H}_{t}, (Ht,S^t,C^t)(H_{t},\hat{S}_{t},\hat{C}_{t}), S⋆S^{\star}, δ\delta, kk
2: β0,β1,ws,wc,wn\beta_{0},\beta_{1},w_{s},w_{c},w_{n}
3: 𝒜t←∅\mathcal{A}_{t}\leftarrow\emptyset
4: for H′∈ℋtH^{\prime}\in\mathcal{H}_{t} in parallel do
5:    S^′,C^′←Evaluate​(H′,𝒟evolve,k)\hat{S}^{\prime},\hat{C}^{\prime}\leftarrow\textsc{Evaluate}(H^{\prime},\mathcal{D}_{\mathrm{evolve}},k)
6:    Δ​S←S^′−S^t\Delta S\leftarrow\hat{S}^{\prime}-\hat{S}_{t}; Δ​C←(C^′−C^t)/C^t\Delta C\leftarrow(\hat{C}^{\prime}-\hat{C}_{t})/\hat{C}_{t}
7:    ν←νt​(H′)\nu\leftarrow\nu_{t}(H^{\prime}) ⊳\triangleright new structural component types
8:    if Δ​S>δ\Delta S>\delta then
9:     c←[ΔC≤β0+β1ΔS]c\leftarrow[\Delta C\leq\beta_{0}+\beta_{1}\Delta S]
10: ⊳\triangleright gain-dependent cost rule, Eq. (7)
11:    else
12:     c←[wsΔS−wcΔC+wnν>0]c\leftarrow[w_{s}\Delta S-w_{c}\Delta C+w_{n}\nu>0]
13: ⊳\triangleright within-band rule, Eq. (17)
14:    end if
15:    g←DomainGuard​(Ht,H′)g\leftarrow\textsc{DomainGuard}(H_{t},H^{\prime})
16:    if S^′≥S⋆−δ\hat{S}^{\prime}\geq S^{\star}-\delta and cc and gg then
17:     𝒜t←𝒜t∪{H′}\mathcal{A}_{t}\leftarrow\mathcal{A}_{t}\cup\{H^{\prime}\}
18:    end if
19: end for
20: Ht+1←arg⁡maxH′∈𝒜t⁡S^′H_{t+1}\leftarrow\arg\max_{H^{\prime}\in\mathcal{A}_{t}}\hat{S}^{\prime}, or HtH_{t} if 𝒜t=∅\mathcal{A}_{t}=\emptyset
21: S⋆←max⁡(S⋆,S^t+1)S^{\star}\leftarrow\max(S^{\star},\hat{S}_{t+1})
22: record each measured edit with a=1a=1 iff its candidate is Ht+1≠HtH_{t+1}\neq H_{t}
23: return Ht+1H_{t+1}

C.2 Proposal-Side Bookkeeping

Atomic edit representation.

A candidate applies a subset of the round’s proposed atomic edits. The budget caps this subset’s size; the possible edit pool changes each round and remains unrestricted across the run.

Equation 9
‖zt‖0≤bt.\|z_{t}\|_{0}\leq b_{t}. (9)

Edit history and component-level summaries.

Every edit in a measured candidate inherits that candidate’s score and token changes. Only edits belonging to the selected round winner receive an acceptance flag. Component summaries track whether a type has been exercised and its best recent measured gain. This bookkeeping becomes more interpretable as fewer edits are bundled.

Equations 10–11
ℒt={(ti,ℓi,hi,di,Δ​Si,Δ​Ci,ai):i≤nt},ai∈{0,1},\mathcal{L}_{t}=\{(t_{i},\ell_{i},h_{i},d_{i},\Delta S_{i},\Delta C_{i},a_{i}):i\leq n_{t}\},\qquad a_{i}\in\{0,1\}, (10)
𝒯t={ℓi:i≤nt},gt(ℓ)=max{ΔSi:ℓi=ℓ,t−ti≤nprune},max∅=−∞.\mathcal{T}_{t}=\{\ell_{i}:i\leq n_{t}\},\qquad g_{t}(\ell)=\max\{\Delta S_{i}:\ell_{i}=\ell,\;t-t_{i}\leq n_{\mathrm{prune}}\},\qquad\max\emptyset=-\infty. (11)

Structured exploration state.

The component vocabulary covers prompts, control flow, configuration, output handling, context management, tools, skills, memory, and subagents. A three-round stall reserves a proposal slot for component types with no measured edit history.

Equations 12–13
𝒦={\displaystyle\mathcal{K}=\{ prompt,control_flow,config,output_plumbing,\displaystyle\texttt{prompt},\texttt{control\_flow},\texttt{config},\texttt{output\_plumbing}, (12)
context_mgmt,client_tool,skill,memory,subagent}.\displaystyle\texttt{context\_mgmt},\texttt{client\_tool},\texttt{skill},\texttt{memory},\texttt{subagent}\}.
ℰt=(σt,𝒰t,mdraft),σt=𝟙[S^t−S^t−w≤δ],𝒰t=𝒦∖𝒯t,\mathcal{E}_{t}=(\sigma_{t},\mathcal{U}_{t},m_{\mathrm{draft}}),\qquad\sigma_{t}=\mathbb{1}[\hat{S}_{t}-\hat{S}_{t-w}\leq\delta],\qquad\mathcal{U}_{t}=\mathcal{K}\setminus\mathcal{T}_{t}, (13)

Structural pruning.

A previously exercised component becomes a pruning target when no recent edit associated with it has a strictly positive gain. The proposer is asked to remove related unproductive machinery in future candidates.

Equation 14
ℬt={ℓ∈𝒯t:gt​(ℓ)≤0}.\mathcal{B}_{t}=\{\ell\in\mathcal{T}_{t}:g_{t}(\ell)\leq 0\}. (14)

C.3 Selection-Side Bookkeeping

Noise-adjusted floor.

After leakage screening, candidates must stay within the calibrated tolerance of the running best accepted score. This stability floor is mandatory before any cost or novelty trade-off.

Equation
S^​(H′)≥S⋆−δ.\hat{S}(H^{\prime})\geq S^{\star}-\delta.

Novelty used by the within-band rule.

Novelty counts distinct tool, skill, memory, and subagent component types that a candidate touches before any edit of that type has previously won. Prompt, configuration, context, and control-flow changes receive no novelty bonus.

Equations 15–16
𝒦str={client_tool,skill,memory,subagent}\mathcal{K}_{\mathrm{str}}=\{\texttt{client\_tool},\texttt{skill},\texttt{memory},\texttt{subagent}\} (15)
νt(H′)=∑ℓ∈𝒦str[ℓ∈comp(H′)∧Nt(ℓ)=0].\nu_{t}(H^{\prime})=\sum_{\ell\in\mathcal{K}_{\mathrm{str}}}\mathbb{1}\!\left[\ell\in\mathrm{comp}(H^{\prime})\ \wedge\ N_{t}(\ell)=0\right]. (16)

Acceptance when the gain exceeds the noise tolerance.

For improvements larger than the noise tolerance, a linear allowance links permitted relative token growth to the measured score gain. It is a cost gate, not a squared-norm penalty.

Equation
Δ​C≤β0+β1​Δ​S.\Delta C\leq\beta_{0}+\beta_{1}\Delta S.

Acceptance when the gain does not exceed the noise tolerance.

For gains within the noise band, a separate weighted rule combines score change, lower token cost, and structural novelty. Coding assigns zero weight to small score changes, so they cannot qualify alone; workspace and engineering runs permit positive score weight. The text says these weights appear in Table 5, but the supplied table omits them, leaving the exact within-band settings unspecified in the paper.

Equation 17
ws​Δ​S−wc​Δ​C+wn​νt​(H′)>0.w_{s}\Delta S-w_{c}\Delta C+w_{n}\nu_{t}(H^{\prime})>0. (17)

Domain-specific non-compensatory guards.

Engineering candidates also fail if valid-output rate drops by more than 0.03 or no-submission rate rises by more than 0.02. Coding and workspace runs impose no additional domain guard.

Final round selection.

The selector chooses the largest measured score among candidates passing the floor, the applicable cost rule, and domain guards. If none passes, it retains the incumbent and updates no accepted edit path.

Appendix D Experiments

D.1 Hyperparameter Setting

Parameters are chosen without consulting held-out tasks. Coding and workspace evolution run for 20 rounds with two trials per task; engineering uses 40 rounds with four trials. Initial edit limits are four, three, and four respectively, decreasing toward one. The noise tolerances are 0.017, 0.004, and 0.020 in fractional-score units. The complete published table preserves exploration, pruning, and gain-dependent cost settings.

Hyperparameter Role Coding Agentic workspace Engineering design
TT evolution rounds 20 20 40
kk trials per task per evaluation 2 2 4
δ\delta empirical noise tolerance 0.017 0.004 0.020
bminb_{\min} final-round edit budget 1 1 1
bmaxb_{\max} initial edit budget 4 3 4
ww stall-detection window 3 3 3
mdraftm_{\mathrm{draft}} reserved exploratory proposals 1 1 1
nprunen_{\mathrm{prune}} pruning window 4 4 5
β0\beta_{0} base cost allowance 0.10 0.10 0.15
β1\beta_{1} gain-dependent cost allowance 44.5 35.4 24.4
Table 5: Hyperparameters used by RRSI in each evolution setting. All choices are fixed without consulting held-out or OOD benchmarks.

Appendix E Qualitative Case Study

Case studies show why superficially similar changes can have different outcomes. An early coding audit is accepted with a 3.93-point gain; another is rejected with a 1.69-point gain and 26.1% added cost. A cheaper candidate loses 2.81 points and fails the stability floor. An engineering directory-recovery fix increases passes from 122/244 to 128/244 for 1.6% more tokens. These illustrate decisions from the reported runs, rather than independently proving each mechanism’s causal contribution.

Domain / Round Harness change Outcome What it illustrates
Coding, R0-A Adds a bounded pre-completion verification audit and guidance for non-blocking polling of long-running jobs. Accepted: +3.93+3.93 points on the evolve set. A reusable behavioral mechanism can justify a relatively broad early-round update when the gain exceeds the noise threshold.
Coding, R0-B Adds a similar verification reminder and long-running-work guidance, but with a smaller measured gain and additional inference cost. Rejected by cost rule: +1.69+1.69 points, +26.1%+26.1\% cost. An apparent improvement is not automatically retained when it lies within the noise band and requires substantial additional computation.
Coding, R8-B Pins the original task instruction into the completion gate so that the policy re-checks the literal specification before submission. Rejected by floor: −2.81-2.81 points despite −13.6%-13.6\% cost. Lower cost alone cannot compensate for a candidate whose performance falls below the noise-adjusted acceptance floor.
Engineering, R2 Adds a bounded recovery hint for the recurring “workdir must be an existing directory” tool-use error. Accepted: 122/244→128/244122/244\rightarrow 128/244 passes, +1.6%+1.6\% tokens. The search can retain small, task-agnostic control-flow fixes that improve reliability with little added complexity.
Table 6: Representative harness-evolution decisions from RRSI. The examples show that evolution is not driven by score alone: candidate specificity, evaluation stability, and inference cost jointly determine whether a change is retained.

Author affiliations

The authors are affiliated with Google Cloud AI Research, the University of North Carolina at Chapel Hill, Stanford University, and Washington University in St. Louis. Peng Xia contributed while a Student Researcher at Google Cloud AI Research.

[Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image][Uncaptioned image]
Concept demonstration

Open image ↗