Figure 1: AgentMercury synthesizes diverse, scalable, and verifiable business environments at scale. (a) The synthesized environment spans 14 industries and 50 countries, covering 4,783 environments with anonymized to prevent hallucination. (b) Comparison of environment scale and multi-service depth with existing agent environments (Trivedi et al., 2024; Wu et al., 2026; Shi et al., 2026; Dong et al., 2026); AgentMercury occupies the regime that is both large-scale and multi-service. (c) Each environment contains rich, stateful structure, including multiple services, tools, state tables, and deterministic SQL-based verifiers that enforce cross-service constraints.
Table 1: Result on EnterpriseOps-Gym (Malay et al., 2026). We evaluate whether training on environments and tasks synthesized by AgentMercury improves agent performance across model scales and policy optimization methods. Results are reported for both Qwen3.5-4B and Qwen3.5-35B-A3B models, with GRPO (Shao et al., 2024) and SAO (Hou et al., 2026) considered as alternative RL algorithms. † signifies that the result are derived from EnterpriseOps-Gym.
Model
Teams
CSM
Email
ITSM
Calendar
HR
Drive
Hybrid
Avg.
GPT-5 †
26.3
36.4
49.0
18.9
41.3
17.9
34.0
23.5
30.9
Gemini-2.5-Pro †
39.3
11.6
31.1
13.9
12.5
4.9
27.0
19.6
19.9
Kimi-K2-Thinking †
30.0
7.1
51.0
12.2
15.4
8.2
39.6
15.7
22.4
Qwen3-4B (Think) †
24.0
3.8
38.4
5.6
5.8
7.1
21.9
15.8
15.3
Qwen3-30B (Think) †
22.0
5.4
51.9
6.7
18.3
7.6
25.7
15.7
19.1
Qwen3-235B (Think) †
28.0
4.7
38.1
9.3
15.7
7.8
23.8
17.7
18.1
Qwen3.5-4B
20.8±2.5
9.2±2.3
23.9±3.0
6.8±1.0
10.4±2.5
9.5±2.8
6.2±1.6
11.7±1.7
12.3±0.2
Qwen3.5-4B + GRPO + Ours
23.0±1.6
5.6±1.5
33.3±0.9
7.4±1.1
13.1±1.6
10.8±4.3
15.6±3.1
17.0±1.1
15.7±0.6
Qwen3.5-35B-A3B
33.9±3.8
7.6±2.5
53.2±2.3
14.6±1.0
18.6±0.9
11.4±1.5
35.9±5.4
23.5±1.7
24.8±0.6
Qwen3.5-35B-A3B + GRPO + Ours
39.8±4.1
11.1±0.9
53.8±7.5
15.6±1.6
21.9±1.9
17.7±0.9
41.2±3.9
23.7±1.1
28.1±1.4
Qwen3.5-35B-A3B + SAO + Ours
39.1±2.3
12.5±2.8
54.8±2.1
16.5±0.5
22.6±0.5
17.7±1.4
39.1±4.4
24.1±2.4
28.3±1.5
Figure 2: From scenario-grounded world construction to agent interaction. (A) Planet transforms a high-level business scenario σ into an executable world w, constructing its company identity, service graph, state schema, seeded initial state s0, and world-level invariants ℛ. A task is then instantiated from the resulting world through task-specific state seeding and a rubric. (B) Within the constructed world, a policy acts from the interaction history, producing actions that deterministically transition the environment from st to st+1, while observations ωt+1 are exposed through the observation interface. The resulting trajectory is evaluated post-episode using the task rubric and, where applicable, the world-level invariants. The formulation thus separates world construction, task instantiation, interaction, and evaluation.
Table 2: Out-of-domain evaluation across mathematical reasoning, coding, scientific computing, tool use, and agentic benchmarks. We report mean ± standard deviation over three independent evaluation runs (N=3). Qwen3.5-4B and Qwen3.5-35B-A3B are evaluated before and after RL training on tasks synthesized by AgentMercury.
Model
AIME26
HMMT
LCB
SciCode
Tau3- Airline
Tau3-
Airline
Tau3- Retail
Tau3-
Retail
Tau3- Telecom
Tau3-
Telecom
BFCL
GPQA- Diamond
GPQA-
Diamond
Tau3-
Airline
Tau3-
Retail
Tau3-
Telecom
GPQA-
Diamond
Qwen3.5-4B
45.9±1.7
28.5±1.2
36.6±2.8
22.6±0.1
48.8±2.1
70.4±1.0
92.5±1.0
30.3±1.8
76.5±0.7
Qwen3.5-4B + GRPO + Ours
56.0±1.8
35.4±1.6
44.0±1.2
25.7±0.7
58.7±0.8
73.6±1.3
91.9±1.3
31.7±1.5
77.5±0.6
Qwen3.5-35B-A3B
91.0±0.3
77.0±0.9
74.3±2.2
29.7±1.5
39.1±10.6
52.7±22.1
49.1±49.8
31.1±1.3
82.8 ±3.0
Qwen3.5-35B-A3B + GRPO + Ours
91.9±2.1
80.0±0.7
79.0±0.5
29.9±1.7
52.5±5.2
57.1±8.3
68.9±20.5
42.5±0.3
85.0±0.5
Qwen3.5-35B-A3B + SAO + Ours
92.2±1.9
83.3±3.1
78.6±1.0
28.3±1.9
50.9±8.1
55.7±10.2
65.5±23.8
42.1±0.1
84.0±1.0
Figure 3: Out-of-domain benchmark performance across training checkpoints for Qwen3.5-4B + GRPO + Ours. The dashed line denotes the base-model performance, while error bars show the standard deviation over three independent evaluation runs. The gradual improvement across heterogeneous benchmarks indicates that the policy learned from AgentMercury environments acquires transferable capabilities beyond the training environments.
Table 3: Authoring executable worlds from high-level business briefs. We report oracle-pass rates over 30 held-out briefs. A world is counted as successful only when it passes all 12 structural validators. Zero-shot provides only the business brief, while Recipe additionally provides an invariant digest and a trimmed construction exemplar.
Model
Zero-shot
Recipe
Claude Opus 4.8
83.3%
80.0%
GPT-5.4
66.7%
66.7%
DeepSeek-V4-Pro
83.3%
86.7%
GLM-5.2
80.0%
76.7%
Mean (API models)
78.3%
77.5%
Qwen3.5-35B-A3B
3.3%
20.0%
Qwen3.5-35B-A3B + AgentMercury
83.3%
10.0%
Figure 4: Training dynamics of Qwen3.5-4B + GRPO + Ours. We report the raw reward, response length, truncated-response ratio, and degenerate-response ratio over training steps. The increasing reward together with decreasing truncation and near-zero degeneration indicates that policy improvement is not accompanied by substantial response collapse or pathological generation.
Table 4: Composition of the synthesized task corpus used for RL. Statistics are computed over all 43,300 tasks.
Statistic
Value
Tasks
43,300
Executable worlds / companies
4,326
Task seeds per world
10
Unique industries
2,287
Unique tools
842
Unique state tables
222
Service combinations
148
Tools per world
10–26 (16.1 avg.)
Services per task
1–5 (2.9 avg.)
Assertions per task
2–10 (5.4 avg.)
Figure 5: Overview of the synthesized environments, sampled training footprint, and task lifecycle in AgentMercury. (A) The full corpus contains 43,300 investigation tasks instantiated from 4,326 synthetic company environments, spanning 2,287 industries, 842 tools, 222 state tables, and 148 service combinations. (B) Across 200 RL training steps, the agent observes 3,200 tasks (7.4% of the full task corpus) while retaining broad coverage of environments, industries, tools, state tables, and service combinations through shuffled sampling. (C) Each training instance follows a structured lifecycle from a high-level business scenario to executable world construction, task instantiation, multi-step tool interaction, deterministic post-episode verification, and reward-based policy optimization. The resulting framework separates world construction from task sampling, allowing a single synthesized environment to support multiple seeded tasks while providing verifiable learning signals for agent training.
Table 5: Coverage of the tasks that received gradients during the reported RL run relative to the full synthesized corpus. These statistics used only for GRPO algorithm.
Statistic
Full
Trained
Coverage
Tasks
43,300
3,200
7.4%
Environments
4,326
2,313
53.5%
Industries
2,287
1,438
62.9%
Tools
842
638
75.8%
State tables
222
137
61.7%
Service combinations
148
93
62.8%
Figure 6: Structural analysis of the synthesized task space. (a) A t-SNE projection of task descriptions based on TF-IDF features shows substantial textual clustering, but the clusters do not correspond to meaningful task metadata such as domain, persona, or task shape. (b) In contrast, projecting the same tasks using their exposed tool sets reveals clear domain-specific structure, indicating that the primary source of task variation lies in the executable tool surface rather than in task language. (c) Distribution of nearest-neighbor cosine similarity in the TF-IDF space, showing that although the overall task space is broad, most tasks still have a highly similar counterpart. (d) The number of exposed tools takes only five values, {10,14,18,22,26}, revealing a discrete task-shape lattice induced by the core tool set and the number of additional resources.
Table 6: Model-parallelism and memory configuration used for Qwen3.5-4B policy optimization.
Parameter
Value
Tensor parallelism
2
Pipeline / context / expert parallelism
1 / 1 / 1
Sequence parallelism
Enabled
Recomputation
Full, uniform, 1 layer
Attention backend
FlashAttention
Attention softmax precision
FP32
Dropout
0
Max tokens per GPU
24,576
Figure 7: Training dynamics of Qwen3.5-35B-A3B with GRPO and AgentMercury. We report the raw reward, response length, truncated-response ratio, and degenerate-response ratio throughout training. The increasing reward is accompanied by a steady reduction in truncation, while degenerate responses remain negligible.
Table 7: Rollout and fully-asynchronous execution configuration.
Parameter
Value
Max response length
16,384
SGLang context length
24,576
Rollout temperature
1.0
End-of-turn token
<|im_end|>
Partial rollout
Enabled
Reasoning / tool parser
Qwen3 / Qwen3-Coder
SGLang memory fraction
0.8
Max running requests
64
Fully asynchronous training
Enabled
Actor / rollout GPUs
2 / 6
Max staleness
4
Rollout in-flight cap
64
Minimum fresh-token ratio
0.75
Weight update interval
1
Fault tolerance
Enabled
Figure 8: Training dynamics of Qwen3.5-35B-A3B with SAO and AgentMercury. We report the raw reward, response length, truncated-response ratio, and degenerate-response ratio throughout training. SAO exhibits a sustained reward increase together with a substantial reduction in truncation, while degenerate responses remain close to zero.
Table 8: Configuration of the executable MCP environments used during policy optimization.
Parameter
Value
Maximum MCP turns
20
Maximum generation tokens / turn
8,192
Arena backend
Local (in-process)
Probe tool
Calendar
Environment creation throttle
8
Ready timeout
600 s
Reset timeout
180 s
Table 9: Reward and verification configuration used during RL training.
Parameter
Value
Reward implementation
AgentMercury rl_reward.py
Reward aggregator
Fraction
Behavior penalty
Enabled
Reward judge
DeepSeek-V4-Flash
Judge temperature
0.0
Judge max tokens
4,000
Truncated-sample handling
Retained during cold start
Table 10: GRPO and optimizer configuration for Qwen3.5-4B.
Parameter
Value
Advantage estimator
GRPO
Surrogate
Decoupled-PPO (icepop)
ϵ-clip / ϵ-clip-high
0.20 / 0.28
Entropy coefficient
0
GRPO std normalization
Disabled
Per-token loss
Enabled
Global batch size
128
Rollout batch size
16
Samples per prompt
8
Dynamic sampling filter
Zero-std reward groups dropped
Optimizer
Adam
β1 / β2
0.9 / 0.98
Learning rate
1×10−6
Learning-rate schedule
Constant
Warmup
0
Weight decay
0
Gradient accumulation
FP32
Table 11: Model and corpus configuration for the Qwen3.5-4B RL experiment.
Agents learn to act through interaction with environments, yet the environments used for training are often manually constructed or synthesized around predefined tasks and benchmarks. This task-centric paradigm makes it difficult to scale environments that reflect realistic and evolving workflows where diverse tasks can naturally emerge from the underlying world. We introduce AgentMercury, a scalable framework for synthesizing executable environments from high-level business scenarios. Rather than constructing an environment for a specific task, AgentMercury first instantiates a persistent world with entities, services, tools, state, and executable cross-service invariants, from which diverse tasks and interaction trajectories can subsequently emerge. We construct 4,783 executable environments spanning 14 industries and 50 countries, and use them as training substrates for reinforcement learning. Despite being generated without targeting the evaluation benchmarks, policies trained on these business-oriented environments improve substantially on both enterprise workflows and out-of-domain benchmarks spanning reasoning, coding, scientific computing, and tool use. In our experiments, Qwen3.5-4B improves from 12.3 to 15.7 on EnterpriseOps-GYM and from 45.9 to 56.0 on AIME26 after training on AgentMercury environments. We further show that the construction process itself can be learned: fine-tuning Qwen3.5-35B-A3B on construction traces increases executable-world authoring success from 3.3% to 83.3% on held-out business scenarios. These results show that scenario-grounded environments can provide useful and generalizable learning signals beyond benchmark-specific training, while their construction can itself become a learnable capability.
