FlowEvo: Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skills

arXiv:2607.215962026-08-19

An AI agent turns its own past successful task-solving routines into reusable skills, getting better over time without any training

LLM agents typically build a fresh procedure (workflow) for every task, and even successful ones are usually thrown away afterward. FlowEvo instead compiles successful workflows into executable skills, stores them in a persistent bank, and later either runs a matching skill directly or feeds it in as guidance while building a new workflow. Using only GPT-4o-mini with no parameter updates, it reaches 85.6% on ALFWorld, 26.4 points above the strongest baseline, while using about one third the tokens.

What they did

When the agent solves a task successfully, FlowEvo automatically compiles the workflow into a reusable skill (a callable piece of code plus usage guidance) and stores it in a skill bank
For a new task, a matching stored skill can either be executed directly or supplied only as contextual guidance while a new workflow is generated
Each skill's real-world usefulness is tracked continuously, and skills that turn out to hurt performance (negative transfer) are automatically suppressed
With a single shared GPT-4o-mini backbone, FlowEvo scored highest among 8 baseline methods on the full standard test splits of ALFWorld, HumanEval, MBPP, GSM8K, and MATH-500
Across 10 base models ranging from 7B to 671B parameters, FlowEvo beat the prior method ExpeL in 49 out of 50 model-dataset comparisons

Figure 1: Overview of FlowEvo. Successful workflows are compiled into executable skills and stored in a persistent bank. Retrieved skills can either be executed directly or supplied as structured context for subsequent workflow generation.

Table 1: Routing modes in FlowEvo. The distinction between direct execution and skill-conditioned generation is central to the mechanism analysis.
Route	Role of retrieved skill	Executes code	Indirect effect
Dynamic generation	No retrieved skill is used.	✗	✗
Direct skill execution	A retrieved skill is called as an executable subroutine before verification.	✓	✗
Skill-conditioned generation	Retrieved skills are supplied only as structured context for planning and generation.	✗	✓

Figure 2: Skill accumulation dynamics on ALFWorld. (a) FlowEvo’s token cost drops sharply once the first compiled skill becomes available around episode 10 (dashed line) and remains substantially below ReAct for the rest of the run. (b) Cumulative direct-reuse count (green) grows nearly linearly to 101/134, and the reuse hit rate (orange) stabilizes around 75%.

Table 2: Main results across three benchmarks (GPT-4o-mini). Bold = best accuracy or lowest tokens; underline = second best.
	ALFWorld	HumanEval	GSM8K
Method	SR (%)	Tokens	pass@1 (%)	Tokens	Solve (%)	Tokens
Reflexion	52.2	31,900	92.7	890	96.8	555
ExpeL	46.3	32,958	89.0	883	92.1	552
ADAS	53.0	29,671	82.4	3,776	90.8	2,820
AFLOW	59.2	30,137	94.7	3,826	93.5	2,494
FlowEvo	82.8	12,267	95.1	880	97.1	541

Figure 3: Per-task-type success rate on ALFWorld (GPT-4o-mini; n per type varies). FlowEvo improves over ReAct on all five displayed task types. The magnitude of improvement tracks how often the compiled skill completed the task through direct execution alone without fallback to skill-conditioned or dynamic generation. The pick_two task type is omitted because its compiled skill was suppressed by skill curation as a case of negative transfer (see Section 4.3).

Table 3: Core decision thresholds used in FlowEvo. All values are defaults; no per-benchmark tuning was performed for the reported results.
Constant	Value
Retrieval
top_k	3
retrieval_threshold	5.0
Historical positive-transfer cap	2.5
Negative-transfer risk cap	3.0
Routing
_COMPATIBILITY_DIRECT_THRESHOLD	0.55
Direct-execute compat gate	≥1.5
_CODE_EXCERPT_THRESHOLD	4.6
Max seed context skills	2
Admission
banned_imports	{os, subprocess, socket, requests, pathlib}
banned_calls	{eval, exec, compile, __import__, open}
prune_audit_fail_threshold	2
Curation (ALFWorld)
_CONTRASTIVE_MIN_GUIDED	5
_CONTRASTIVE_MIN_UNGUIDED	3
_CONTRASTIVE_HARM_THRESHOLD	−0.1
_AUDIT_MIN_USES	3
_AUDIT_UTILITY_THRESHOLD	0.5
_AUDIT_INACTIVITY_LIMIT	50 episodes
Adaptive escalation
Level 1 (greedy)	temp=0.0, max=2048
Level 2 (mini-ensemble, 2 candidates)	temp=0.5, max=2048
Level 3 (retry, hot)	temp=0.7, max=2048
Level 4 (retry, cool)	temp=0.2, max=2048
Reflexion reasoning	temp=0.0, max=300
Execution
ALFWorld max_steps	50
Sandbox timeout (code/math)	10 s

Table 4: Active skills after 134 ALFWorld episodes. Uses counts episodes in which the skill was retrieved above retrieval_threshold and thus participated in routing (whether the router eventually selected direct execution or skill-conditioned generation, or downgraded to dynamic). Success / Fail are the outcomes of those retrieval-matched episodes; Utility = Success / Uses. Entries are cumulative over the run. For five of the six task types, Uses equals Direct+Cond from Table 5; look_at_obj_in_light has 3 additional retrieval matches that were routing-downgraded to pure_dynamic (see Section 4.4: "the retrieved skill matched on 15 of 18 episodes").
Task type	Status	Uses	Success	Fail	Utility
pick_and_place_simple	active	23	23	0	1.00
pick_clean_then_place_in_recep	active	28	28	0	1.00
pick_heat_then_place_in_recep	active	21	21	0	1.00
pick_cool_then_place_in_recep	active	20	18	2	0.90
look_at_obj_in_light	active	15	13	2	0.87
pick_two_obj_and_place	suppressed	10	1	9	0.10

Table 5: Internal routing modes by task type on ALFWorld. “Direct” = pure direct_skill; “Cond.” = direct_then_* plus exemplar_guided; “Dyn.” = pure_dynamic. “Direct SR” is the success rate among pure direct-execute episodes. “Overall SR” is the pass rate on all episodes of the task type.
Task type	n	Direct	Cond.	Dyn.	Direct SR	Overall SR
look_at_obj_in_light	18	0	12	6	—	14/18 (78%)
pick_and_place_simple	24	14	9	1	14/14	24/24 (100%)
pick_clean_then_place_in_recep	31	22	6	3	22/22	29/31 (94%)
pick_cool_then_place_in_recep	21	12	8	1	12/12	19/21 (90%)
pick_heat_then_place_in_recep	23	19	2	2	19/19	22/23 (96%)
pick_two_obj_and_place	17	0	10	7	—	3/17 (18%)
Total	134	67	47	20	67/67	111/134 (83%)

Why it matters

It shows agents can accumulate useful experience purely at inference time, improving both accuracy and cost efficiency without retraining the underlying model. That matters for real-world settings with repetitive tasks, such as household robot control or automated code/math problem solving, where cutting inference cost while raising accuracy has direct practical value.

Terms in this paper

workflow · the reasoning, tool-use, and code-execution procedure an agent constructs on the fly to solve a task
skill bank · a persistent store of executable procedures compiled from previously successful workflows
negative transfer · when reusing a stored skill actually lowers success rate compared to not using it
skill-conditioned generation · using a stored skill only as reference context to guide building a new workflow, rather than running it directly
training-free · improving agent behavior without updating the underlying model's parameters

Original abstract (English)

Large language model agents can adapt to complex tasks by constructing workflows at inference time, but procedures discovered in one episode are usually discarded after execution. Existing skill libraries provide reusable executable routines, but are typically assembled offline and do not grow from the agent's own workflows. We introduce FlowEvo, a training-free framework in which workflows and skills co-evolve at inference time. FlowEvo compiles successful workflows into callable skills, stores them in a persistent bank, and uses retrieved skills either through direct execution or as context for constructing new workflows. It also tracks each skill's downstream utility and suppresses skills that cause negative transfer. Using a shared GPT-4o-mini backbone, FlowEvo achieves the highest accuracy among 8 baselines on the full standard splits of ALFWorld, HumanEval, MBPP, GSM8K, and MATH-500. On ALFWorld, it reaches 85.6%, 26.4 points above the strongest baseline, while using roughly one third as many tokens. Across 10 base models spanning 7B to 671B parameters, FlowEvo outperforms ExpeL in 49 of 50 model-dataset comparisons. Code is available at https://github.com/DEFENSE-SEU/FlowEvo.

Authors · Zeyu Ren

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Zeyu Ren et al., arXiv:2607.21596, arxiv-nonexclusive