Figure 1: The evaluation feedback modality determines the ceiling of skill evolution. Left: single-turn QA patches the gaps visible in the first exchange; the evolution gradient then decays and the TSR curve saturates around round 2. Right: multi-turn interaction keeps exposing latent failures—each round of revision lets the dialogue proceed further and reach the next layer of defects—so the gradient renews itself and TSR keeps climbing.
Table 1: Baselines and variants.
Method
Skill source and update mechanism
Multi-turn
Rounds
Original Skill
Hand-authored initial Skill, never updated
No
0
Self-Reflection
Model self-reflects and edits the Skill directly, no evaluation feedback
No
4
Single-turn QA
Single-turn QA evaluation-driven evolution (11)
No
4
SkillEvo
Iterative evolution driven by simulated multi-turn interaction
Yes
4
Figure 2: Overview of SkillEvo. The upper layer generates trustworthy feedback to drive revision; the lower layer governs structural degradation so that the knowledge carrier remains intact.
Table 2: Per-round TSR of each method on the evaluation set (%).
Method
Init
R1
R2
R3
R4
Original Skill
30.0
—
—
—
—
Self-Reflection
30.0
59.2
58.7
57.4
58.8
Single-turn QA
30.0
58.9
64.5
65.7
66.4
SkillEvo
30.0
59.4
71.3
77.9
81.8
Table 3: Ablation (evaluation-set TSR, %).
Variant
Overall TSR
SkillEvo (Full)
81.8
(a) Single-turn QA
66.4
(b) w/o Governance
78.6
Table 4: Dual-sided orthogonal evaluation (%).
Side / property
Metric
Value
Simulator, coverage
cU (intent coverage)
98.9
Simulator, fidelity
ρ (human-rated similarity)
95.3
Agent, accuracy
sC (exposed-intent accuracy)
71.1
Table 5: Cross-round regression rate RegR (%).
R1→2
R2→3
R3→4
First-to-last change
28.2
24.4
21.1
−7.1
Table 6: Knowledge bloat (cumulative growth relative to S0, %).
Setting
Cumulative bloat
Remark
With governance (SkillEvo Full)
+2.8
Growth concentrates in the first round and then tapers off
Without governance
+16.2
Bloat accumulates round after round with no dissolution mechanism
Table 7: Evaluation scenarios and dataset.
Scenario category
Skill
Cloud service
#Tickets
Marketing
dianshi-consultation
Dianshi campaign platform
400
Dev & collaboration tools
code-assistant-consultation
CodeBuddy coding assistant
400
Storage
cos-consultation
Cloud Object Storage (COS)
200
Storage
cbs-consultation
Cloud Block Storage (CBS)
160
Dev & collaboration tools
cloudbase-consultation
CloudBase
200
AI & LLM platforms
tokenhub-consultation
TokenHub LLM service platform
200
AI & LLM platforms
tencent-adp-consultation
Agent Development Platform (ADP)
200
Networking & edge
edgeone-consultation
EdgeOne edge security acceleration
200
Compute
scf-consultation
Serverless Cloud Function (SCF)
40
Total
9 Skills
6 categories
2,000
Table 8: Model assignment and evolution parameter configuration.
Component
Parameter
Value
Skill Editor (Generator)
Model family
A (deepseek-v4-pro)
Verifier / Attributor / User Agent / Governor (Evaluator)
Agent Skills are today either hand-authored or produced in a single LLM generation pass, and consequently possess no closed loop through which they might improve from the interaction failures they actually cause. Recent work does close this loop, but derives its feedback from single-turn question-answering evaluation. The consequence is a sharp asymmetry: once the first round has patched the gaps that a single exchange can reveal, the evolution gradient decays, the defects that surface only across multiple turns remain invisible, and evolution stalls. Governance in these systems is likewise driven by an end-to-end verification score, a scalar gate that can reject a degraded candidate but can neither localize nor repair its structural cause. We argue that the binding constraint on sustained skill evolution is neither editing capability nor the number of iterations, but whether the evaluation feedback keeps supplying trustworthy evolution gradients. We introduce SkillEvo, in which trustworthy feedback generates the gradient and controllable governance constrains its direction. The first component recasts multi-turn user simulation from an evaluation endpoint into a feedback generator: follow-up questions expose defects layer by layer, so that every round of revision both consumes feedback and produces new feedback. The second replaces the passive rejection of a scalar gate with an independent governance layer that actively repairs factual degradation and structural bloat, preventing the gradient from drifting as degradation accumulates. Across six categories of cloud services, 9 production Skills, and 98 skill-reference files, SkillEvo surpasses self-reflection-based evolution by 23.0 points and single- turn-QA-driven evolution by 15.4 points.
