Training Agents to Evolve with Their Harness: TaoLive Digital Avatar Agent Technical Report
arXiv:2608.157632026-08-21
Training a compact live-stream AI avatar so it keeps up when its own operating rules keep changing
A live e-commerce AI avatar host must answer product questions instantly while marketing rules and reply strategies get rewritten frequently by operations teams. This paper separates an editable Harness (skills, tools, prompts, hooks) from fixed model weights, and introduces Harness-Aware Training (HAT) so a small, fast model stays accurate even as that Harness keeps evolving. Deployed in Taobao Live's digital-avatar service, the trained model met strict latency targets while substantially improving reply quality.
METAL LAB explanatory visual
Decoupling the Harness from the model so each can evolve independently
Evidence statusMeasured results reported
Harness EvolutionEditing Skills, prompts, tools, and Hooks while keeping model weights fixed, letting operators change behavior within hours
HSA (state augmentation)Deliberately varying skill names/content, tool schemas, prompt structure, and Hook behavior during training data construction
HAT three-stage trainingHSA-SFT on diverse Harness configurations, then General OPD to restore general skills, then HSA-RL in a simulated live-room environment
Production runtimeThe trained compact model and the continuously evolving Harness meet together to serve live viewer requests
An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.
What they did
Large models understand a rewritten Harness zero-shot but are too slow for live interaction, while compact models are fast but overfit to whatever fixed Harness they were trained on, breaking when that configuration changes.
The Harness is a set of editable modules (Skills, tools, prompt pipeline, Hooks) kept separate from model weights, so operators can update behavior in hours via 'Harness Evolution' without retraining.
Harness-State Augmentation (HSA) deliberately varies skill identifiers/content, tool schemas, prompt structure, and Hook behavior during training so the model learns to read the actual current configuration instead of memorizing fixed names or templates.
Training runs in three stages: HSA-SFT learns from teacher trajectories generated under varied Harness configurations, General On-Policy Distillation restores general capabilities lost during domain fine-tuning, and HSA-RL uses reinforcement learning in a production-informed simulator under augmented Harness states.
On Live-Stream QA the HAT model scores 94.8 (versus base model 80.3 and the strongest general LLM tested at 93.0), on Harness-Variant QA it scores 94.6 (versus base 75.4), and on a single H20 GPU it reaches P50 latency 3.4s and P95 8.1s.
Figure 1: Application flow and the two independent update paths. Harness Evolution edits agent runtime modules while keeping model weights fixed; Harness-Aware Training (HAT) provides a well-performing, low-latency, and robust model. Both paths meet inside the same production runtime.
Table 1: Approximate intent mix in the available internal traffic summary.
Scenario
Proportion
Product Q&A
∼46%
Casual chat and engagement
∼19%
Clarification follow-up
∼16%
After-sales handling
∼7%
Discount and promotion inquiry
∼4%
Presentation-order adjustment
∼2%
FAQ-based reply
∼2%
Silent refusal of irrelevant content
<1%
Figure 2: Harness Evolution with a fixed DeepSeek-V4-flash on dev-set (n=482). The process is non-monotonic: Evolution 2 is the selected development checkpoint, while later long-tail edits introduce regressions.
Table 2: Evaluation sets and sources. The sets cover real live-stream reply quality, in-family Harness variation, synthetic tool and prompt robustness, general instruction following, and actual deployment latency.
Set
Size
Source
Evidentiary role
𝒯1
978
Real live-room
Primary industrial quality (live-stream reply).
𝒯2
978
Real live-room, aug. Harness
In-family robustness to new Harness contexts.
𝒯3
2023
Synthetic avatar scenarios
In-domain transfer under broader tools/prompts.
𝒯4
541
Public IFEval
General instruction following (official evaluator).
𝒞judge
482
Real live-room, human-labeled
Judge calibration, not a policy test set. Also dev-set for harness evolving.
𝒟perf
110
Deployment replay
End-to-end deployment latency.
Figure 3: Simplified Harness Agent runtime control flow. Prompt rules and dynamically loaded Skills assemble the model context; lifecycle Hooks validate inference, tool use, and stopping. A failed stop gate starts another round, while a passed gate produces the final response and structured trajectory. Per-call fallback and trajectory-level recovery protect different failure scopes.
Table 4: Training-pipeline ablation. Each row adds one stage to the previous checkpoint; ↑/↓ denotes a descriptive score change from the row above and does not include retraining variance.
Configuration
𝒯1
𝒯2
Tool
Prompt
IFE-P
IFE-I
Non-augmented pathway (single-run trajectory)
Base
80.3
75.4
69.5
72.8
81.5
87.7
+SFT
89.5↑9.2
88.2↑12.8
82.0↑12.5
68.2↓4.6
73.8↓7.7
82.4↓5.3
+General OPD
89.5
89.1↑0.9
84.3↑2.3
72.6↑4.4
82.3↑8.5
87.9↑5.5
+RL
95.1↑5.6
94.4↑5.3
83.7↓0.6
66.7↓5.9
82.7↑0.4
87.8↓0.1
Augmented pathway (single-run trajectory)
+Aug-SFT
90.0↑9.7
90.2↑14.8
85.2↑15.7
77.3↑4.5
81.7↑0.2
87.4↓0.3
+General OPD
90.9↑0.9
90.9↑0.7
85.6↑0.4
77.1↓0.2
81.9↑0.2
88.0↑0.6
+RL
94.1↑3.2
93.7↑2.8
84.3↓1.3
78.7↑1.6
83.0↑1.1
88.2↑0.2
+AUG Env (Ours)
94.8↑0.7
94.6↑0.9
84.0↓0.3
77.6↓1.1
83.5↑0.5
88.7↑0.5
Figure 4: Overview of HAT. (a) HSA constructs diversified Harness states; HSA-SFT followed by General OPD produces the shared initialization. (b) Controlled comparison of Agentic RL and Agentic OPD under the original Harness and HSA environments. Every variant starts from the HSA-SFT + General OPD result, and each OPD variant receives its paired same-environment RL teacher. The selected Agentic RL policy under HSA is HSA-RL.
Table 5: Single-run comparison of Agentic optimization variants. All configurations share the same Aug-SFT + General OPD initialization; ⋆ marks the deployment choice rather than statistical superiority.
Method
Environment
𝒯1
IFE-P
PR-AVG
Agentic RL
Original
94.1
83.0
78.7
Agentic RL
Augmented
94.8⋆
83.5⋆
77.6
Agentic OPD
Original
94.0
82.8
73.9
Agentic OPD
Augmented
93.9
81.7
75.5
Paired contrasts
Δ(RL−OPD)
Original
+0.1
+0.2
+4.8
Δ(RL−OPD)
Augmented
+0.9
+1.8
+2.1
Δ(Aug−Orig)
RL
+0.7
+0.5
−1.1
Δ(Aug−Orig)
OPD
−0.1
−1.1
+1.6
Figure 5: Checkpoint trajectories across four training configurations formed by crossing HSA at the SFT and RL stages, evaluated on Accuracy, Effectiveness, Skill Selection, and Tool Rationality. In these trajectories, HSA-SFT is associated with higher reply-quality rewards, while HSA-RL is associated with stronger agentic-behavior rewards, especially Skill Selection and Tool Rationality; combining HSA-SFT and HSA-RL gives the most balanced late-stage reward profile.
Table 6: Single-run isolation of data augmentation at SFT and RL stages. The reported differences are descriptive and do not include retraining variance.
SFT Stage
RL Stage
𝒯1 AVG
IFE-P
ΔIFE
Standard
Standard RL
95.1
71.3
–
Standard
Augmented RL
94.9
72.3
+1.0
Augmented
Standard RL
95.4
80.7
+9.4
Augmented
Augmented RL
93.3
81.3
+10.0
Augmented SFT: +9.4 IFE-P; augmented RL: +1.0.
Figure 6: Single-run CoT-length diagnostics for the selected HSA-RL trajectory. Faint lines show recorded step values and bold lines show 25-step trailing means. Panel (a) separates tool-call and final-reply CoT lengths and marks the no-penalty (L≤100), linear (100<L<200), and saturated (L≥200) regions. Panel (b) shows the logged score q(L)∈[0,1] and the equivalent raw reward contribution −0.1q(L). The figure is a training diagnostic, not a causal ablation of latency or quality.
Table 7: Controlled low-concurrency deployment replay. Latencies are seconds; Decode is the arithmetic mean of client-observed completion tokens/s over model calls. It is length-sensitive and is not an intrinsic cross-checkpoint kernel-speed measure. TTFT is the first call’s client-observed P95 and includes network and queueing. Each row contains 100 measured Agent requests after 10 warm-up cases. API rows are operational references, not same-hardware model comparisons.
Configuration
MTP
C
Wall P50
Wall P95
TTFT P95
Decode
≤15s
Success
Vendor API routes
DeepSeek-V4-flash
–
1
11.210
21.191
2.067
103.44
71%
99%
DeepSeek-V4-flash
–
2
11.976
26.278
1.291
97.01
71%
98%
DeepSeek-V4-pro
–
1
14.312
28.882
1.312
53.53
61%
99%
DeepSeek-V4-pro
–
2
13.752
23.393
1.248
55.04
60%
99%
Qwen3.6-35B-A3B · one H20
Base
Off
1
4.101
10.172
0.508
195.42
100%
100%
Base
Off
2
4.560
11.504
0.623
138.81
97%
100%
Base
On
1
3.648
9.805
0.502
215.95
99%
100%
Base
On
2
4.806
10.251
0.698
165.74
97%
100%
Task-trained checkpoint (Ours) · one H20
Ours
Off
1
4.176
8.98
0.499
160.30
100%
100%
Ours
Off
2
4.936
10.074
0.692
129.22
100%
100%
Ours
On
1
3.407
8.114
0.553
271.40
100%
100%
Ours
On
2
4.479
9.047
0.754
196.15
100%
100%
Figure 7: Judge alignment to a human-labeled calibration cohort (n=482). Accuracy and Effectiveness agreement improve through evidence-tool, rubric, and voting revisions; the cohort evaluates the scoring instrument rather than the policy.
Table 8: Harness-Aware Benchmark held-out test results before and after Harness edits. Counts are total detected errors; error reduction is computed relative to the before-edit count.
Model
Before
After
Error reduction
Naive SFT
149
122
18.1%
Ours
238
115
51.7%
Figure 8: Single-run standalone MTP-loss trajectories for a task-trained policy with a transplanted and subsequently post-trained NextN head, and for the base model with its native factory-trained head. Faint lines show step-level loss; bold lines show a 25-step trailing mean. The runs have different lengths and provide adaptation diagnostics only: they do not compare convergence speed, training stability, sample efficiency, or compute efficiency.
Table 9: Human blind-test preference distribution on 100 paired real live-streaming requests.
Preference
Count
Share
Harness better
35
35.0%
Tie
64
64.0%
ReAct better
1
1.0%
Table 10: Attribution of the 35 Harness-preferred examples. Categories are assigned from annotator rationales after the blind decision.
Attribution category
Count
Share
More accurate input understanding
12
34.3%
More reliable output
8
22.9%
More appropriate scenario behavior
8
22.9%
Higher response quality
5
14.3%
More appropriate tool use
2
5.7%
Table 11: Preliminary seven-day online A/B point estimates in Taobao Live’s production digital-avatar business. Metrics are relative changes of the Harness system against the ReAct control.
Metric
Harness vs. ReAct
Cumulative confirmed-receipt GMV per bucketed user
+5.54%
Completed orders per bucketed user
+0.80%
Table 12: Working inventory of built-in and MCP-discovered runtime interfaces.
Retrieve campaign conditions and buyer-side resources from the external marketing service.
MCP calculation
calculate_optimal_promotion, calculate_promotion
Compute eligible promotional combinations and resulting prices.
Table 13: Working Skill inventory and routing responsibilities.
Reply Skill
Response responsibility
Strategy Skill
Tool-chain responsibility
item_qa
Product attributes, specifications, comparison, price, and recommendation.
tool_strategy_benefit
Entitlement, price, and promotion retrieval.
chillchat
Non-product conversation and engagement.
tool_strategy_compare
Multi-product comparison and retrieval.
aftersale
Returns, exchanges, complaints, and other after-sales requests.
tool_strategy_multi
Aggregation and orchestration for multiple comments.
clarification
Follow-up questions when available context is insufficient.
tool_strategy_switch
Presentation-order adjustment.
faq_reply
Responses grounded in configured FAQ entries.
general_discount
Room-level promotion retrieval and response composition.
refusal
Declines for inappropriate or unsupported requests.
change_order
Confirmation of presentation-order changes.
thanks
Responses to gratitude and positive feedback.
greet
Welcome messages for viewers entering the room.
Table 14: Fixed-policy Harness Evolution record. Values come from six direct evaluation summaries on the same 482-item dev-set. “Selected” is an engineering early-stop decision.
Stage
Harness changes
Acc.
Eff.
Engineering diagnosis
ReAct
Conventional reasoning–action loop without the modular Harness.
80.33
84.58
System baseline.
Harness base
Initial manually maintained Skills and modular runtime.
82.40
87.16
Establishes the editable Harness.
Evolution 1
Add refusal tool and stop-loop Hook; rewrite refusal, chat, and after-sales Skills; add four global constraints.
92.13
84.16
Accuracy rises sharply; internal diagnosis attributes Effectiveness loss to over-triggered refusal.
Evolution 2
Add seven whitelist exclusions before refusal; map explanation triggers and require the relevant tool call.
92.55
92.75
Selected; restores Effectiveness while retaining Accuracy.
Long-tail rules interact and regress both metrics.
Evolution 4
Relax length, tool-trigger, parameter, transaction-intent, and attribution restrictions.
91.51
89.96
Partial rollback does not recover Effectiveness.
Table 15: Independent paired-bootstrap stability check. Differences are reported in percentage points; intervals are 95% percentile bootstrap intervals over evaluation items.
Contrast / dimension
Δ [95% CI]
Pboot(Δ≤0)
IFEval prompt-level accuracy
Naive SFT − Base
−6.47[−10.72,−2.22]
0.9992
Ours − Base
+2.03[−1.11,+5.18]
0.1134
Ours − Naive SFT
+8.50[+4.44,+12.38]
0.0000
Prompt Robustness
Ours − Naive SFT
+8.46[+5.27,+11.66]
<0.0001
Naive SFT − Base
−3.22[−6.59,+0.21]
0.9661
𝒯1 Live-Stream QA: Ours − Base
Accuracy
+6.95[+4.60,+9.41]
<0.0001
Effectiveness
+30.88[+27.76,+33.90]
<0.0001
AVG
+18.92[+17.00,+20.81]
<0.0001
Table 16: Same-checkpoint 𝒯1 quality check with MTP Off and On (n=978). Both rows use the same Final Evaluation Judge. AVG is the arithmetic mean of Accuracy and Effectiveness.
Serving mode
Accuracy
Effectiveness
AVG
MTP Off
95.8
93.8
94.8
MTP On
96.22
94.22
95.22
Table 17: MTP trade-off across concurrency. Arrows show Off → On. Decode TPS is client-observed mean per call; Wall and TTFT are P95 seconds. C=1, 2, and 8 are single runs. C=4 values are means of three run-level statistics, not pooled percentiles.
Checkpoint
C
Decode TPS
Speedup
Wall P95 (s)
TTFT P95 (s)
≤15s
Qwen3.6 Base
1
195.42→215.95
1.11×
10.172→9.805
0.508→0.502
100→99%
2
138.81→165.74
1.19×
11.504→10.251
0.623→0.698
97→97%
4
105.04→113.25
1.08×
13.408→16.266
0.643→1.088
97.3→92.3%
8
70.28→70.19
1.00×
17.848→26.136
0.953→1.519
90→79%
Ours
1
160.30→271.40
1.69×
8.98→8.114
0.499→0.553
100→100%
2
129.22→196.15
1.52×
10.074→9.047
0.692→0.754
100→100%
4
99.65→126.64
1.27×
10.869→12.460
0.692→1.349
99.3→98.0%
8
59.01→70.22
1.19×
18.342→20.298
0.900→2.574
91→75%
Table 18: C=1 workload shape. Values are means over successful complete-Agent requests or their constituent model calls.
Route / checkpoint
MTP
Calls/req
Input/call
Output/call
DeepSeek-V4-flash API
–
4.04
6,052
146
DeepSeek-V4-pro API
–
3.67
5,537
119
Qwen3.6 Base
Off
3.23
5,622
115
Qwen3.6 Base
On
3.08
5,571
109
Ours
Off
3.04
5,679
106
Ours
On
3.12
5,786
105
Table 19: MTP serving configuration used for the reported quality check and deployment measurements.
Setting
Engine argument
Value
Speculative algorithm
--speculative-algo
NEXTN
Speculative steps
--speculative-num-steps
3
EAGLE top-k
--speculative-eagle-topk
1
Draft tokens
--speculative-num-draft-tokens
4
Single-token acceptance threshold
--speculative-accept-threshold-single
0.5
Accumulated acceptance threshold
--speculative-accept-threshold-acc
0.7
Table 20: SGLang log-sample diagnostics for MTP-On serving. Triples are mean/P50/P95; Queue is P95/max.
Checkpoint
Running req
Queue
Accept len
Accept rate
Qwen3.6 Base
2.03/1/8
0/0
2.94/2.92/3.58
.735/.730/.890
Ours
2.03/1/7
0/0
3.12/3.12/3.62
.780/.780/.910
Findings
On the real-world Live-Stream QA benchmark, HAT reached an average score of 94.8, above the untuned base model's 80.3 and the strongest general LLM tested at 93.0.
On Harness-Variant QA, which tests robustness to changed Harness configurations, HAT reached 94.6 versus the base model's 75.4.
Fixed-Harness SFT lowered general instruction-following (IFEval) by 7.7 points from the base model, but HAT avoided this regression and reached 83.5 on IFEval.
On a single NVIDIA H20 GPU with optimization enabled, the system achieved P50 latency of 3.407s and P95 latency of 8.114s.
Online A/B testing in Taobao Live's production digital-avatar service showed positive point estimates for GMV and item-page views over a preliminary seven-day window.
Where it can be used
Live e-commerce or customer-service agents where marketing rules, reply strategies, or compliance requirements change frequently and retraining is too slow
Tool-using agentic chatbots that dynamically load skills or tools, where robustness to renamed tools or restructured prompts is needed to avoid brittle behavior
Latency-critical real-time services that want to make a compact model more robust through training-time environment variation rather than test-time adaptation
Limits and open work
The reported numbers are specific to Taobao Live's product and live-streaming domain, a Qwen3.6-35B-A3B-scale model, and a single H20 GPU setup, so transfer to other domains or model sizes is not demonstrated.
Latency results were measured at low concurrency (C=1-2); at higher concurrency (C=4-8) the MTP speedup drops and P95 no longer improves, so behavior under heavier production traffic is not fully characterized.
The MTP draft-head transplant experiments are described by the authors as single-run engineering diagnostics, not claims about convergence speed or stability across multiple seeds.
The online A/B test results are described as preliminary seven-day point estimates, so longer-term or larger-scale production stability is not yet confirmed.
Why it matters
Services that need both split-second responses and frequent strategy changes usually have to trade one for the other, and this work shows a concrete way to keep a fast, small model robust to those changes by putting the operating environment itself into the training data. It also reports a real production deployment with positive A/B results, not just offline numbers.
Terms in this paper
Harness · The editable runtime configuration of Skills, tools, prompts, and Hooks that governs agent behavior independently of model weights
HSA (Harness-State Augmentation) · A data-augmentation technique that rewrites skill names/content, tool schemas, prompt structure, and Hook behavior to expose the model to varied configurations during training
HAT (Harness-Aware Training) · The three-stage training pipeline (HSA-SFT, General OPD, HSA-RL) designed to make a compact model robust to a changing Harness
GRPO/GDPO · Reinforcement-learning optimization methods that compute advantages by comparing groups of sampled response trajectories
MTP draft head · A multi-token-prediction module that speculatively predicts several future tokens ahead to speed up inference
Original abstract (English)
AI-powered digital avatar streamers must answer product questions, engage viewers, and execute marketing strategies in real time, demanding low latency, frequent strategy updates, and accurate yet effective responses. Evolvable Harnesses, whose Skills, Hooks, prompts, and tools can be updated independently of model weights, enable rapid iteration but expose a trade-off: large models adapt zero-shot yet are too slow, whereas compact models meet latency targets but overfit to fixed Harness configurations. We propose Harness-Aware Training (HAT), which trains compact models to adapt to changing Harnesses. Its key component, Harness-State Augmentation (HSA), applies task-preserving transformations to Skill identifiers and content, tool schemas, prompt structures, and Hook functions. Training proceeds in three stages: HSA-SFT learns reasoning and tool use from strong-model trajectories across diverse environments; General On-Policy Distillation restores generalization lost during SFT; and HSA-RL improves robustness to changing Harnesses through reinforcement learning in augmented environments. Across four evaluation sets, HAT achieves 94.8 on Live-Stream QA (base: 80.3; strongest general LLM: 93.0) and 94.6 on Harness-Variant QA (base: 75.4). Unlike Fixed-Harness SFT, which lowers IFEval by 7.7 points from the base model, HAT avoids this regression and reaches 83.5. On one NVIDIA H20 GPU, the optimized system delivers P50 and P95 latencies of 3.4 s and 8.1 s. Deployed in Taobao Live's digital-avatar service, it also yields positive online A/B test results for GMV and item-page views.
![Figure 6: Single-run CoT-length diagnostics for the selected HSA-RL trajectory. Faint lines show recorded step values and bold lines show 25-step trailing means. Panel (a) separates tool-call and final-reply CoT lengths and marks the no-penalty (L≤100), linear (100<L<200), and saturated (L≥200) regions. Panel (b) shows the logged score q(L)∈[0,1] and the equivalent raw reward contribution −0.1q(L). The figure is a training diagnostic, not a causal ablation of latency or quality.](https://media.metallab.ai/papers/2608.15763/f5.svg)
