AI news and explainers at 7 AM weekdays, plus a Sunday weekly at 8Get it in your inbox›
AgentRoom: Concurrent Multi-Agent Coding in a CRDT-Backed Shared Workspace
arXiv:2608.237402026-08-26
Letting multiple AI coding agents share one workspace and coordinate in real time beats running them one-by-one or in uncoordinated parallel
AgentRoom is a protocol that lets several coding AI agents work simultaneously on a shared file system backed by a CRDT (a data structure that merges concurrent edits without conflicts), using file-claim, status, and broadcast tools. Across five frontier coding models on four backend coding tasks, two coordinated agents abandoned fewer tasks than a single agent and produced more consistent results run to run. The authors argue that coordination itself, not parallelism or the CRDT merge mechanism alone, is what drives the improvement.
METAL LAB explanatory visual
AgentRoom's coordination structure and six-condition comparison
Evidence statusMeasured results reported
Shared CRDT workspaceN coding agents share one file system; pycrdt (a Yrs-based CRDT) automatically merges concurrent edits at the character level.
Six compared conditionsQuality scores ordered: ChatDev-style sequential (0.333) < parallel-merge (0.456) < Solo (0.544) < shared-only (0.575) < collaboration without MCP (0.588) < AgentRoom (0.669).
Abandonment suppressionSolo's common early-quit failure was reduced 13.7-fold in odds when AgentRoom coordination was used, pooled across 12 model-task combinations.
Agent-count scalingQuality peaked at 2 agents then dropped at 3 and 4, while message traffic grew super-linearly beyond 3 agents.
An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.
What they did
The researchers built AgentRoom, where multiple AI coding agents work concurrently on one shared file system, with coordination features (file claim, status check, message broadcast) exposed as MCP tools (a standard interface letting models call external tools).
Five frontier coding CLI models (Claude Sonnet 4.6, Haiku 4.5, Codex GPT-5.4, etc.) ran backend tasks — authentication systems, a marketplace API, a financial ledger, a trading platform — under six conditions: Solo, uncoordinated parallel-merge, ChatDev-style sequential handoff, and AgentRoom.
A solo agent often abandoned hard tasks after producing just one stub file; pooling all 12 model-by-task combinations with a Cochran-Mantel-Haenszel test found Solo's odds of abandonment were 13.7 times higher than AgentRoom's (95% CI 3.9-48, p<10^-5).
At matched compute, AgentRoom scored 0.213 higher on average quality than uncoordinated parallel execution (Welch t=3.35, p=0.003), and also beat a condition with the CRDT and collaboration prompt but no MCP coordination tools, pointing to the coordination tools themselves as the key driver.
Scaling the number of agents from 1 to 4 showed quality peaking at 2 agents then declining, while message traffic grew super-linearly past 3 agents, suggesting the single broadcast channel may become a bottleneck.
Figure 1: T4 six-condition coordination ablation: AgentRoom is the top condition, while concurrency without coordination (parallel-merge) and sequential hand-off (ChatDev-style) fall below the single-agent baseline.
Table 1: T4 Sonnet 4.6 six-condition ablation (LLM-judge composite, budget-fair 30–700 s pool). Parallel-merge vs AgentRoom: +0.213, Welch t=3.35, 𝐩=0.003. The shared+collab-noMCP probe ran under a 1200 s wall budget; its runs are filtered to the same 30–700 s envelope as every other cell (Appendix B.13). Cross-model replication in Appendix B, Table 8.
Condition
Mean
n
σ
Note
ChatDev-style (sequential 3-phase)
0.333
6
0.197
sequential paradigm baseline
Parallel merge (2 indep + union)
0.456
12
0.178
concurrent, no coordination
Solo (1 agent)
0.544
32
0.222
single-agent baseline
Shared only (2 agents, no MCP)
0.575
11
0.155
shared CRDT, no signalling
Shared + collab prompt, noMCP
0.588
7
0.188
+ prompt, no MCP tools
AgentRoom (2 agents, full)
0.669
14
0.140
CRDT + room MCP + collab prompt
Figure 2: AgentRoom architecture: N coding agents share a workspace mediated by a CRDT; the MCP server exposes claim, release, broadcast, state, read.
Table 2: Cross-model T4 ES with bootstrap 95% CI (10 000 resamples), AST-scored subsample of the full pool (≥30 s elapsed). n is smaller than the LLM-scored pools because the AST scorer covers a subsample. Headline scorer is AST (TypeScript compiler API); regex-scorer values after the slash for cross-check. ∗ CI excludes 1. Two CLI-unstable models (Gemini, GPT-5.4-mini) excluded (see §4.4). Reported as exploratory conjecture: only Haiku T4 clears the CI under the AST scorer.
Model
SWE-bench V.
Solo (n)
×2 (n)
ES (AST / regex)
95% CI (AST)
Haiku 4.5
73.3%
0.520 (13)
0.929 (8)
1.79× / 1.58×
[1.24, 3.35]∗
Codex GPT-5.4
∼80%
0.248 (9)
0.487 (13)
1.96× / 1.07×
[0.85, 8.85]
Sonnet 4.6
79.6%
0.463 (11)
0.630 (15)
1.36× / 1.14×
[0.84, 2.29]
Figure 3: Real-time CRDT merge during a single T4 Sonnet AgentRoom ×2 run. The MCP-mediated chat (left) drives concurrent character-level edits in the shared file (right); both α’s and β’s insertions co-exist on the same line via the Yrs CRDT, with no byte-level conflict raised in this run. Per-message Bezier leaders link each chat broadcast to the code line it talks about.
Table 3: Sonnet 4.6 ES across the difficulty gradient (regex/AST scorer, outlier-filtered ≥0.3).
Task
Solo (n)
×2 (n)
ES
T1
0.919 (4)
0.910 (7)
0.99×
T2
0.840 (3)
0.904 (4)
1.08×
T3
0.887 (3)
0.943 (4)
1.06×
T4
0.796 (10)
0.958 (10)
1.20×
T5
0.697 (7)
0.810 (7)
1.16×
Figure 4: Solo (slate) vs AgentRoom ×2 (rust) score distributions for the three CLI-stable models on T4 (LLM-judge composite, budget-fair pool). σ contracts by ∼30–45% under AgentRoom in every panel (Sonnet 0.23→0.14, Haiku 0.25→0.17, Codex 0.27→0.15); mean shift varies by model.
Table 4: Heterogeneous vs. homogeneous ×2 on T4 (LLM-judge, budget-fair). Tests is the mean over all budget-fair runs in the cell (n=17/4/3), which is not the LLM-scored n shown in the Score column.
Pair
Score (n)
Tests
Sonnet ×2 (homo)
0.669 (14)
32.8
Sonnet+Codex
0.721 (3)
16.0
Sonnet+Gemini
0.542 (3)
22.0
Figure 5: Agent-count scaling on T4 Sonnet 4.6 (N=1–4, budget-fair pool). Left: mean LLM-judge composite with σ band peaks at N=2 then declines. Middle: mean tests passing peaks at N=3. Right: AgentRoom message count rises super-linearly past N=3, consistent with a single-broadcast-channel coordination overhead.
Table 5: Wall vs compute on T4 Sonnet (LLM-judge composite, budget-fair pool).
Condition
N
Wall (s)
Compute (s)
Quality
Solo
1
550
550
0.544
Parallel merge
2
558
1077
0.456
Shared only
2
601
1201
0.575
AgentRoom ×2
2
561
1072
0.669
AgentRoom ×3
3
566
1572
0.553
Figure 6: Cochran-Mantel-Haenszel forest plot of Solo-vs-AgentRoom 1-file abandonment across all 12 model×task strata (Sonnet/Haiku/Codex × T1/T2/T4/T5).
Table 6: T5 (trading platform, 900 s budget), AST scorer, per-model cells. All CLI-stable cells are point-estimate-positive on ×2 except Codex; CIs do not exclude 1 at this n. The Gemini row is CLI-unstable and informational only (§4.4); sparse heterogeneous cells are in the text above.
Model
Solo (n)
×2 (n)
×3 (n)
ES(×2)
Tests (×2)
Haiku 4.5
0.416 (8)
0.707 (7)
0.822 (1)
1.70×
38
Sonnet 4.6
0.618 (7)
0.674 (7)
0.730 (2)
1.09×
57
GPT-5.4
0.667 (9)
0.531 (7)
0.761 (1)
0.79×
8
Gemini Flash (unstable)
0.851 (3)
0.506 (9)
0.246 (3)
0.59×
12
Figure 7: Bootstrap effect-size distributions (10 000 resamples) for the Tier III cross-model cells. Only Haiku T4 has its 95% CI strictly above 1; the other five cells span 0.79–1.96× point-estimate with CIs that include 1.
Table 7: Frequency of unprompted collaborative behaviors across 6 valid ×2 AgentRoom runs on T4 Sonnet.
Behavior
Runs exhibiting (of 6)
Module claiming via broadcast
6/6
Plan adjustment after reading teammate
5/6
Export list shared for teammate
4/6
Proactive gap filling
3/6
Cross-agent bug fix
2/6
Apology for file boundary violation
2/6
Figure 8: Cost–quality Pareto on T4. Shaded region is Pareto-dominated; dotted lines are iso-efficiency (quality/cost) contours. 2×Haiku AgentRoom clears 1×Sonnet at matched plotted cost.
Table 8: Cross-model T4 ablation (LLM-judge composite, budget-fair pool). For Sonnet and Haiku, AgentRoom is the top condition and parallel-merge falls below solo. Codex non-Solo/AgentRoom cells were not collected (vendor-CLI hang).
Baselines and partial bundles
Ours
Model
Solo
Parallel merge
Shared only
noMCP
AgentRoom ×2
Sonnet 4.6
0.544 (n=32)
0.456 (n=12)
0.575 (n=11)
0.588 (n=7)
0.669 (n=14)
Haiku 4.5
0.530 (n=35)
0.383 (n=12)
0.657 (n=7)
0.579 (n=6)
0.662 (n=17)
Codex GPT-5.4
0.384 (n=12)
–
–
–
0.529 (n=17)
Figure 9: Single representative T4 Sonnet AgentRoom ×2 run. Top swim-lane: agent α broadcasts (timestamps in-card); middle: per-file ownership Gantt (rust = α, slate = β); bottom: agent β. Bezier S-curve leaders link each chat broadcast to its time on the Gantt. The whole protocol (claim, conflict-detect, hand-off, completion broadcast) emerges from the advisory six-step prompt (§2.3) without any orchestrator.
Table 9: Threshold sensitivity for the Tier I CMH. Each row is a threshold combination; cells report Solo/AgentRoom abandonment counts per model and the resulting CMH common OR with 95% CI and p-value.
Thresholds
Sonnet 4.6
Haiku 4.5
Codex GPT-5.4
CMH pooled
d thr
f thr
Solo
Room
Solo
Room
Solo
Room
OR
95% CI
p
<150
≤1
6/33
0/17
12/35
1/17
9/17
0/21
24.2
[3.5, 169]
0.00001
<150
≤2
6/33
0/17
12/35
1/17
9/17
2/21
11.9
[3.0, 47]
0.00005
<150
≤3
6/33
0/17
12/35
1/17
10/17
6/21
6.3
[2.1, 19]
0.00069
<200
≤1
6/33
0/17
12/35
1/17
9/17
0/21
24.2
[3.5, 169]
0.00001
<200
≤2 (paper)
6/33
0/17
12/35
1/17
9/17
2/21
11.9
[3.0, 47]
0.00005
<200
≤3
6/33
0/17
12/35
1/17
10/17
6/21
6.3
[2.1, 19]
0.00069
<250
≤1
6/33
0/17
12/35
1/17
9/17
0/21
24.2
[3.5, 169]
0.00001
<250
≤2
6/33
0/17
12/35
1/17
9/17
2/21
11.9
[3.0, 47]
0.00005
<250
≤3
6/33
0/17
12/35
1/17
10/17
6/21
6.3
[2.1, 19]
0.00069
<300
≤1
6/33
0/17
12/35
1/17
9/17
0/21
24.2
[3.5, 169]
0.00001
<300
≤2
6/33
0/17
12/35
1/17
9/17
2/21
11.9
[3.0, 47]
0.00005
<300
≤3
6/33
0/17
12/35
1/17
10/17
6/21
6.3
[2.1, 19]
0.00069
Figure 10: The two scorers measure the same thing loosely, not interchangeably. Each point is one T4 run carrying both a regex composite and an LLM-judge composite (n=246, the runs scored by both); Pearson r=0.67. The spread off the diagonal is why the body uses one scorer throughout and reports the other as a cross-check rather than averaging them.
Table 10: Pool sensitivity for the T4 Sonnet ablation (LLM-judge composite). The substrate-vs-AgentRoom ordering is preserved under both pools; the budget-fair pool is primary.
Pool
Solo
Parallel merge
Shared only
AgentRoom ×2
30–700 s (paper, budget-fair)
0.544 (n=32)
0.456 (n=12)
0.575 (n=11)
0.669 (n=14)
≥30 s (full pool)
0.464 (n=42)
0.436 (n=13)
0.527 (n=14)
0.639 (n=15)
Table 11: Solo + collaboration-style prompt without teammates, T4, n=3 each (LLM-judge composite). The structured prompt alone is model-dependent: it hurts Sonnet (channel is the load-bearing variable) and helps Haiku (the structured prompt also acts as workflow scaffolding); the channel and prompt-as-workflow are not separately identified for Haiku at this n.
Model
Solo (bare, budget-fair)
Solo + collab prompt (n=3)
AgentRoom ×2 (budget-fair)
Sonnet 4.6
0.544 (n=32)
0.448 (range 0.20–0.62)
0.669 (n=14)
Haiku 4.5
0.530 (n=35)
0.604 (range 0.55–0.66)
0.662 (n=17)
Table 12: DevBench cross-domain validation (Haiku 4.5, 600 s budget). Pass = hidden pytest suite passes; durations in seconds. geotext: ×2 completes within budget while solo times out; stocktrends, textcnn: both fail (task too hard for budget).
Task
Solo (PASS, dur)
×2 AgentRoom (PASS, dur)
arxiv-digest
PASS, 70 s
PASS, 120 s
readtime
PASS, 110 s
PASS, 100 s
chakin
PASS, 130 s
PASS, 130 s
geotext
FAIL, 603 s timeout
PASS, 390 s
hone
PASS, 90 s
PASS, 160 s
hybrid-images
PASS, 100 s
PASS, 170 s
lice
PASS, 330 s
PASS, 270 s
particle-swarm-opt
PASS, 100 s
PASS, 120 s
stocktrends
FAIL, 600 s
FAIL, 480 s
textcnn
FAIL, 601 s
FAIL, 856 s
Pass rate
7/10
8/10
Table 13: Scorer sensitivity: leave-one-out on T4 (Sonnet 4.6, solo vs ×2 AgentRoom). The solo mean reported here (0.786) is computed over a smaller subset (4 runs) than the canonical solo subset used in Table 1; the smaller subset is the intersection of solo runs for which we retained per-dimension intermediate scorer outputs at the time of writing, kept here only as a robustness check. The AgentRoom > solo ordering is preserved under every single-dimension ablation and the ratio stays within 1.12–1.25 on this subset.
Removed dimension
Solo mean
×2 mean
ES
(none, full scorer)
0.786
0.928
1.18
Type Safety
0.721
0.904
1.25
Defensive Coding
0.815
0.944
1.16
Test Quality
0.854
0.956
1.12
Design
0.793
0.921
1.16
Test Pass Rate
0.749
0.916
1.22
Findings
Pooling 12 model-by-task strata, Solo's odds of task abandonment were 13.7 times AgentRoom's (95% CI 3.9-48, p<10^-5, Tarone homogeneity p=0.92).
On the T4 financial-ledger task with Sonnet, AgentRoom (2 agents) beat uncoordinated parallel-merge by 0.213 in mean quality score (Welch t=3.35, p=0.003).
Across six conditions on T4, a clean ordering emerged: ChatDev-style sequential (0.333) < parallel-merge (0.456) < Solo (0.544) < shared-only (0.575) < collaboration without MCP (0.588) < AgentRoom (0.669).
AgentRoom (2 agents) cut run-to-run standard deviation by roughly 30-45% versus Solo (e.g., Sonnet 0.23 to 0.14, Haiku 0.25 to 0.17, Codex 0.27 to 0.15).
Scaling from 1 to 4 agents, quality peaked at 2 agents (0.669) then declined at 3 (0.553) and 4 (0.489), while message volume grew super-linearly beyond 3 agents.
Where it can be used
Can inform the design of multi-agent coding systems building large, multi-file backend projects (APIs, ledgers) by adding explicit coordination protocols.
Could help mitigate the problem of single agents giving up on hard tasks by introducing file-claim and shared-status mechanisms.
Useful as a reference before scaling beyond 2 agents, given the observed communication bottleneck at higher agent counts.
Limits and open work
Experiments are limited to Express.js/TypeScript backend tasks with some cross-checks in Python and Rust; broader generalization to other languages or domains is untested.
Whether the quality decline at higher agent counts stems from a single broadcast channel bottleneck was not confirmed by a controlled ablation and remains an open question.
Some models (Gemini 3 Flash, GPT-5.4-mini) were excluded from headline results due to CLI instability under concurrent MCP execution, described as a deployment issue rather than a property of AgentRoom itself.
The probe isolating the MCP coordination tools' individual contribution used a small sample (n=7) with a confidence interval spanning zero, so the finding is read as an ordering rather than a precise percentage split.
Heterogeneous model pairings (e.g., Sonnet+Codex vs Sonnet+Gemini) showed widely varying results, indicating coordination does not erase capability gaps between models.
Why it matters
For anyone deploying multiple AI coding agents, this gives evidence that adding explicit coordination mechanisms — file claims, shared status — reduces failure rates and run-to-run variance compared to naive parallel execution or sequential role handoffs. However, the effect size varies by model and task difficulty, so it should not be read as a guaranteed performance boost in all settings.
Terms in this paper
CRDT (Conflict-free Replicated Data Type) · A data structure that lets multiple people or programs edit the same data concurrently and automatically merge changes without conflicts, used in real-time collaborative editing tools.
MCP (Model Context Protocol) · A standard interface that lets AI models call external tools or query system state.
1-file abandonment (stub-and-exit) · A failure pattern where an agent creates only a skeleton of one file and exits early, judging the task too hard.
LLM-judge score · A quality rating from 0 to 1 assigned by another language model (here Sonnet 4.6) applying a fixed rubric to the output.
Parallel-merge · A setup where agents work independently in separate workspaces and their files are merged afterward, with no real-time coordination.
Original abstract (English)
Concurrent multi-agent coding promises division of labor across modules, robustness through redundancy, and parallel exploration at the natural granularity of multi-file projects. Realtime collaborative editing protocols solve this coordination problem for human teams via Conflict-free Replicated Data Types (CRDTs), but the LLMs underneath generate one token at a time and existing multi-agent coding systems inherit this serial limit: they either sequence agents through phase handoffs or pool independent samples without coordination, and a single agent abandons up to half of hard tasks with a one-file stub-and-exit. AgentRoom is a realtime collaborative editing protocol for concurrent coding agents. Its runtime layer exposes file-level claim, status, and broadcast as MCP tools on a CRDT-merged shared filesystem. Five frontier coding-CLI models ran four backend coding tasks, with cross-language checks in Python DevBench and Rust+axum. For CLI-stable models, AgentRoom with 2 agents abandons fewer tasks than Solo and has less run-to-run variation. At matched-compute, one positive mean LLM-judge contrast puts AgentRoom over parallel-merge. The other contrast, a bundle probe, puts full AgentRoom above each partial case: an ordering rather than a percentage split. Coordination, not parallelism or CRDT-merge, bears the load.
