AI news and explainers at 7 AM weekdays, plus a Sunday weekly at 8Get it in your inbox›
DarwinX: Evolving Agent Harnesses Through Natural Selection
arXiv:2608.075452026-07-30
An AI agent got stronger by evolving only its prompts and tools, not its underlying model, through a selection process modeled on natural selection
DarwinX keeps the LLM's weights completely frozen and instead evolves the agent's 'harness' -- prompts, tools, skill documents, and control flow -- as a population of variants selected like natural selection. It only promotes variants that solve new tasks without breaking previously solved ones, and merges complementary lineages from an archive. Across four benchmarks covering terminal tasks, web automation, and code repair, it reports roughly a 17-point average gain.
METAL LAB explanatory visual
How DarwinX's selection loop works
Evidence statusMeasured results reported
Frozen modelThe underlying LLM (GPT-5.5, GPT-5.6, or Opus 4.8) keeps the exact same weights throughout every evolution run
Generate harness variantsFailure analysis, teacher demonstrations, and self-contrast signals produce small edits to prompts, tools, and control flow
Preserve-and-extend screeningOnly variants that solve a new task without regressing previously solved ones get promoted
Archive and recombinationEven non-promoted variants are kept; complementary lineages are merged to combine their distinct strengths
Four-benchmark checkGains are tested for whether they hold up on Terminal-Bench 2.1, TerminalWorld, WebArena-Infinity, and transfer to SWE-bench Verified
An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.
What they did
Prior self-improving agents that edit along a single lineage tend to get stuck on early choices or silently regress one task family while fixing another.
DarwinX keeps a population of harness variants in an archive and only promotes a child under a 'preserve-and-extend contract': it must solve something new without losing what the parent already solved.
Three learning signals -- failure analysis, teacher demonstrations, and contrasting the agent's own passing/failing rollouts -- feed into one shared edit interface.
There are no gold-label answers or hand-picked winners; fitness comes purely from each benchmark's own verifier measuring solve rate (avg@k).
On Terminal-Bench 2.1 the evolved harness reached 83.2% (+7.7 points) on a matched base and 84.7% on a stronger base at the verified leaderboard frontier; on TerminalWorld's held-out split it reached 68.3%, ahead of every off-the-shelf agent tested.
Figure 1: With the base model frozen, evolving the harness alone matches or beats the strongest prior agent on four benchmarks. Left: variants survive on measured fitness (avg@k, no gold solutions) and complementary survivors are merged. Right: bars within a panel share one frozen model; the hatched bar is zero-shot transfer of the Terminal-Bench 2.1 harness. y-ranges are truncated and differ per panel.
Table 1: Positioning among self-improving-agent methods (✓ present, ∼ restricted, ✗ absent). All share the same inner loop and differ in the selection wrapped around it: Search and Selection group the two failure modes we target, path dependence and cross-task interference. Appendix Table 7 gives the mechanism behind each mark.
Edits
Search
Selection
Signal
Method
tools & control flow
tools &
control flow
population archive
population
archive
cross-lineage merge
cross-lineage
merge
bounded regression
bounded
regression
noise-aware avg@k
noise-aware
avg@k
teacher & self signals
teacher &
self signals
tools &
control flow
population
archive
cross-lineage
merge
bounded
regression
noise-aware
avg@k
teacher &
self signals
Optimizers over one designated artifact
OPRO/PromptBreeder/TextGrad
✗
∼
∼†
✗
✗
✗
ADAS/AFlow/GPTSwarm
∼
✓
✗
✗
∼
✗
SkillOpt
✗
✗
✗
∼
∼
✗
Agents that edit their own scaffold
SICA
✓
∼
✗
✗
✗
✗
DGM
✓
✓
✗
∼
∼
✗
HarnessX
✓
∼
✗
✓
∼
✗
DarwinX (ours)
✓
✓
✓
✓
✓
✓
Figure 2: DarwinX’s selection loop with the model frozen. Left: the preserve-and-extend contract. Middle: the archive of alternative lineages. Right: shared memory carried across generations.
Table 4: WebArena-Infinity per-application audit-clean pass@1 on the official 10-application, 1,260-task real suite; Δ is Monet (DarwinX)’s gain over base Monet. Baseline provenance and raw pre-audit scores are in Appendix D.3.
Application
Kimi
Qwen
Gemini+BU
GPT-5.5+BU
Monet (base)
Monet (DarwinX)
Δ
Elation clinical records
50.0
54.2
81.7
92.5
95.8
96.7
+0.9
Elation prescriptions
23.3
41.7
80.8
90.8
20.0
95.0
+75.0
GitLab plan and track
39.3
37.1
63.6
77.9
63.6
97.9
+34.3
Gmail
70.0
56.7
75.0
85.0
25.0
98.3
+73.3
Gmail accounts and contacts
40.0
33.3
61.7
87.5
21.7
91.7
+70.0
Handshake career exploration
50.0
50.5
50.5
83.5
36.5
84.0
+47.5
Linear account settings
54.2
65.8
73.3
81.7
43.3
94.2
+50.9
PayPal wallet
70.7
71.4
88.6
90.0
49.3
95.7
+46.4
Superhuman general
15.0
25.8
50.0
80.8
31.7
87.5
+55.8
Xero invoicing
52.5
55.8
80.8
93.3
39.2
96.7
+57.5
Overall
43.3
48.3
69.3
86.1
43.5
93.0
+49.5
Figure 3: DarwinX’s per-generation operators. Left: the mutation loop and the three learning signals that drive it. Middle: variants classified by how their solved set changes, where those preserving inherited solves stay eligible for recombination while the rest contribute only distilled lessons. Right: the merge operator and its acceptance criterion.
Table 6: TB2.1 skill-bundle diff: the seven skills the evolved lineage adds over base Monet, all in one verification / artifact-contract family. Skills are co-selected, so this attributes composition, not per-skill effect.
Evolved skills
Role
verifier-contract contract-candidate
Derive the task’s acceptance contract and check the solution against it before finalizing.
Verify the graded artifact (output file, format, and values) and iterate a fix-and-recheck loop.
real-tool-artifact tool-grounded-artifact
Ground outputs in real tool execution rather than asserted or simulated results.
security-contract-repair
Repair the solution against security and contract checks.
Figure 7: WebArena-Infinity evolution as optimization. (b) shows the same run as a lineage tree: accepted (blue) and reverted (grey) variants, the primary lineage (gold, base → evolved), and recombination edges (dashed).
Table 7: Mechanism-level comparison with the closest self-improving-agent systems, expanding Table 1. “Promotion rule” is the evidence a candidate must produce before it is kept; “cross-task interference” is how each method prevents an edit that helps one task family from silently regressing another.
Method
Editable surface
Search structure
Promotion rule
Cross-task interference
Optimizers over one designated artifact
OPRO, PromptBreeder, TextGrad
Instruction text; tools and control flow stay fixed.
Iterative keep-best, or a genetic population with prompt crossover.
Scalar score on a fixed development set, or a textual gradient from failures.
Not addressed; the search targets a single task or few-shot pool.
ADAS, AFlow, GPTSwarm
The composition graph over otherwise-fixed components.
Archive of past workflows, or MCTS over graph edits.
Mean accuracy on the target benchmark.
One benchmark per search; no per-task preservation check.
SkillOpt
An external skill document.
Single-lineage keep-best, framed as a gradient-descent analogy.
A held-out validation point estimate gates each edit.
One domain at a time.
Agents that edit their own scaffold
SICA
The agent’s own source code.
A single lineage of self-modification.
Benchmark reward on the current coding suite.
One coding domain; the authors report an early-edit plateau.
DGM
Agent source code.
Open-ended archive, stochastic single-parent mutation, no merge operator.
Score against the parent on a task subset that grows with confidence.
Staged subsets, but no explicit preservation contract.
HarnessX‡
A typed harness: prompts, tools, and control flow.
Staged single-lineage pipeline; variants are kept isolated from each other.
Per-edit gate on the average score plus a seesaw test.
Isolation keeps task families apart, so sub-threshold regressions still accumulate.
DarwinX (ours)
The full harness: a skill layer (prompts, memory, distilled knowledge) and a code layer (tools, control flow, agent loop).
Population archive of typed nodes; parents sampled by cumulative lineage gain; complementary specialists merged into inherited children.
Fitness enabler (g>0, R≤δ) adjudicated by a verifier, then avg@k confirmation and a preservation probe before a node may steer search.
Specialists are retained and recombined rather than isolated, and the preservation probe bounds what any promotion may cost.
(b) Archive lineage tree (node size ∝ screening score).
Table 8: Benchmark-specific base models, evolution and reporting protocols. The base model is frozen throughout every matched comparison, so the harness is the only thing DarwinX changes.
Benchmark
Frozen base
Evolution data
Report data
Selection signal
Report metric
TB2.1
GPT-5.5∘
89 verifier tasks
same 89 tasks
avg@3 screen, avg@5 confirm
avg@5
TerminalWorld
Opus 4.8*
94 train tasks
41 held-out tasks
adaptive avg@k subsets
pass@1
WAI
GPT-5.5
300 synthetic intents
1,260 real tasks
LLM judge, avg@3/avg@5
deterministic pass@1
SWE-V (transfer)
Opus 4.8
none (transfer target)
500 issues
n/a (frozen)
official pass@1
Figure 8: Invalid trajectories before vs. after evolution, by application (left) and mechanism (right). Evolution cuts invalid trajectories from 293 to 17, leaving only raw-state mutations.
Table 10: Construction of the 300-intent WAI evolution split. The benchmark’s own task suites are never read at any stage, so deduplication is intra-pool only.
Stage
Intents
Operation
Seeds
1,080
Two models propose intents from each application’s description document
Raw synthesis
11,279
Dual-model expansion of the seeds under per-application quotas (18 applications)
Restricted to the 12 applications with a served instance
Evolution split
300
Balanced draw of 25 intents per served application
Figure 9: Per-application success on the real WAI suite, raw pre-audit (left) vs. audit-clean (right). The audit collapses the base’s inflated successes (→43.5%) while leaving Monet (DarwinX) nearly unchanged (→93.0%).
Table 11: Regex patterns used to detect invalid state access or mutation.
#
Regex (paraphrased)
Catches
1
/apistate/
Any read or write of the /api/state scoring endpoint
Replace local storage with hand-edited serialized state
INVALID_RAW_STATE_MUTATION
Bypasses application semantics
Send a fabricated full state to /api/state
INVALID_EVALUATION_PLANE_ACCESS
Uses benchmark synchronization as a control channel
sqlite3 app.db ’UPDATE patients ...’
INVALID_DIRECT_DATABASE_MANIPULATION
Host access plus a direct database edit
Inject SQL through a request to bypass authorization or alter rows
INVALID_EXPLOIT_OR_PRIVILEGE_ESCALATION
Exploit rather than authorized semantics
Forge an admin token and call an otherwise valid API
INVALID_EXPLOIT_OR_PRIVILEGE_ESCALATION
Exceeds the current user’s authority
Table 13: Contract-oriented browser skills added by evolution.
Skill
When to use
Core procedure
web_task_contract
General browser/web-UI tasks; durable state change or exact filtered/sorted/read answer
Derive an explicit acceptance contract (target, operation, exact final observable state, forbidden substitutions, persistence) → find a valid path, UI first → execute the smallest change → verify both rendered UI and backing state, and reload to confirm persistence → if a target seems missing, prove “not found” from ≥2 independent app surfaces before declaring a no-op.
filtered_list_report_contract
Count / latest / oldest / value questions over lists and tables
Preserve the active collection scope (tab, status, search, project, date range) while applying the requested filter; count across the whole scoped set (not just the rendered page); answer with only the requested value.
browser_spa_state_contract
Durable state changes where visible controls are missing/ambiguous
Derive the exact field-level contract; inspect app-owned stores/reducers/action helpers; mutate through the app’s own action/persistence path; then read back both state and UI.
browser_config_contract
Durable configuration records (filters, rules, reminders, routing)
Prove every field (condition, action, enabled, timing, channel, persistence), not a partial visible match.
Table 14: The evolved browser prompt replaces an absolute UI-only rule with a bounded semantic fallback and persistence verification.
Aspect
Base prompt (before)
Evolved prompt (after)
Interaction policy
“Interact ONLY through the UI…Do NOT write application state directly or touch /api/state.”
“Prefer real UI controls first…If a bounded audit proves no visible UI path can satisfy a durable state-changing task, you may inspect app-owned stores, reducers, loaded modules, public helper methods, and readback paths, then use the app’s own exposed action/update helper for the smallest targeted mutation. Do NOT touch /api/state, write local/session storage, use seed/reset helpers, or call state-sync internals.”
Finishing (verification)
“For a state-changing task, make the change in the UI, screenshot to confirm, then stop.”
“For a state-changing task, verify both app-owned state/readback and the rendered UI; reload or navigate away/back to confirm persistence, then stop.”
Findings
On Terminal-Bench 2.1 (89 tasks), base Monet on GPT-5.5 scored 75.5%; the DarwinX-evolved harness reached 83.2% (+7.7 points), and on a stronger GPT-5.6 base it reached 84.7%, matching or exceeding the public verified leaderboard leader.
On TerminalWorld's 41 held-out tasks (disjoint from the 94 training tasks), Monet (DarwinX) on Opus 4.8 solved 28/41 (68.3%), ahead of every off-the-shelf agent evaluated and up from 25/41 (+7.3 points) for the unevolved base.
On WebArena-Infinity, evolving only on 300 synthetic intents lifted real-task pass@1 on 1,260 unseen tasks from 43.5% to 93.0% after audit-clean scoring.
A Terminal-Bench 2.1-evolved harness transferred unchanged to SWE-bench Verified (500 issues) and produced measured performance there.
A submission audit found no harness-level verifier-gaming, and invalid trajectories dropped from 293 before evolution to 17 after.
Where it can be used
Designing pipelines that improve coding or terminal agents by evolving prompts, tool sets, and control flow instead of retraining the base model.
Organizations that want to convert evaluation compute into durable agent capability without touching model weights.
Building agent-improvement loops in any domain that has its own automated verifier, without needing gold-label answers.
Limits and open work
The approach requires a benchmark with its own automated verifier, which the authors note is often unavailable in real deployment settings.
avg@k scoring requires multiple rollouts per candidate, which is affordable as a periodic offline job but too costly for per-request evolution.
Co-evolving the model and harness together, and generalizing preservation constraints to compliance requirements, are proposed as future directions but not tested here.
How much of a harness survives a base-model swap, and how many generations a warm archive needs for re-selection, has not been measured.
The Terminal-Bench 2.1 skill-bundle attribution reflects the combined effect of co-selected skills, not the isolated causal contribution of any single skill.
Why it matters
It shows that meaningful agent capability gains can come from selecting better prompts and tool configurations rather than retraining model weights, which matters for anyone trying to improve agents cheaply. It also suggests a harness can be treated as a durable asset that carries over across model generations, reducing the cost of model upgrades.
Terms in this paper
harness · the prompts, tools, skill documents, memory, and control flow that surround and direct an LLM in an agent
avg@k · a scoring method that runs the same task k times and averages the success rate
preserve-and-extend contract · a rule that only promotes a harness variant if it solves a new task without meaningfully losing tasks it already solved
recombination · combining edits from two different harness lineages into one child that inherits the strengths of both
pass@1 · a metric measuring whether a task was solved correctly on a single attempt
Original abstract (English)
An LLM agent's capability depends not only on model weights but on its harness: prompts, tools, skills, and control flow. Self-improvement loops already edit harnesses, yet single-lineage search is path-dependent and local wins often regress other tasks. We introduce DarwinX, which treats self-evolution as selection over a population of harnesses with the model frozen: a preserve-and-extend contract admits only variants that extend coverage without regressing, an archive keeps alternative lineages for recombination, and failure-, teacher-, and self-derived evidence share one edit interface. Fitness comes from each benchmark's own verifier: no gold solutions, no hand-picked winners. Across four benchmarks that progressively separate the evolution signal from the test, one loop adds about 17 points on average: Terminal-Bench 2.1 rises +7.7 to 83.2% on a matched base and to the verified frontier at 84.7% on a stronger one; TerminalWorld's held-out split reaches 68.3%, ahead of every off-the-shelf agent; WebArena-Infinity real-task pass@1 rises from 43.5% to 93.0% audit-clean; and a Terminal-Bench 2.1 harness transfers unchanged to SWE-bench Verified. What evolves is general agent competence, not benchmark-specific patches, so it survives changes of task, verifier, and base model. A frozen model need not be a fixed agent: harness selection turns evaluation compute into durable capability.
