工作日早上 7 点读 AI，周日早上 8 点读周报订阅邮件

METAL LAB

MARS: Multi-Specialist LLM Relay System for Competitive Programming

arXiv:2608.239182026-08-26

解竞赛编程题时,让分主题的专家AI接力改代码,比让一个通才模型包办全部角色更高效

MARS用动态规划、图论、字符串、几何等按主题划分的专家智能体,取代了现有多智能体编程系统里笼统的策划者、编码者、调试者角色,每个专家都依靠检索算法理论语料来支撑自己的判断。系统会为每道题挑选一支最多三人的相关专家小队,由起始专家写出C++17初稿,之后每一轮由当值专家用公开样例实际运行代码,再决定保留、修复还是交给下一位专家。在Gemma 4骨干模型上、CodeContests测试集中,MARS达到0.624的通过率,比直接提示高14.4个百分点,以3.3倍更低的耗时缩小了与CodeSIM(0.731)的大部分差距。

METAL LAB 解读图

MARS接力流水线结构

证据状态已报告实测结果

  1. 专家池与组队11个按主题划分的专家智能体依据共享的cp-algorithms检索语料进行自我评估,挑出最多三名相关专家组队。
  2. 起始专家写初稿从小队中选出的起始专家写出初始C++17解决方案。
  3. 执行-自查-交棒轮次每一轮都在沙箱中用公开测试样例运行当前代码,当值专家据此决定保留、修复还是交给下一位专家。
  4. 接力终止条件出现明确停止信号、可用专家用尽、达到8步上限,或连续无进展时接力停止。
  5. 基础设施修复与结果最后由只处理样板代码问题的修复步骤收尾,最终在Gemma 4上取得0.624的通过率。
这是 METAL LAB 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 现有多智能体编程系统通常只把工作拆成策划者、编码者、调试者这类通用角色,把具体算法技巧的选择完全交给底层大模型,缺乏真正注入领域知识的机制。
  2. MARS从11个主题专家池中,通过基于检索的自我评估挑出最多三名相关专家组队;起始专家写出初始C++17方案,此后每一轮都在沙箱里用公开测试样例运行代码,由当值专家决定保留、修复还是交棒给下一位专家。
  3. 在165道CodeContests测试题、Gemma 4骨干模型上,MARS平均每题只经过2.3个记录阶段就达到0.624±0.006的通过率,比直接提示高14.4个百分点,比只用单一专家的Single-RAG高9.5个百分点,比并行集成基线高6.0个百分点。
  4. MARS仍落后于反复进行计划-模拟-调试的更重系统CodeSIM(0.731),但以3.3倍更低的实际运行耗时缩小了大部分差距,且每题token花费的波动明显更小。
  5. 换成其他骨干模型(Qwen3.5-27B、GPT-5.4-mini)或改用Python语言时,Direct < Single-RAG < MARS的排序依然成立;去掉检索增强后通过率下降了2.0个百分点。
Figure 1: MARS relay pipeline. A task is routed from a pool of RAG-grounded topic specialists to a team of at most three agents. Each turn runs code generation, public-test execution, and self-check/handoff; repair code is rerun locally before the current code and relay packet move to the next specialist or final submission.
Figure 1: MARS relay pipeline. A task is routed from a pool of RAG-grounded topic specialists to a team of at most three agents. Each turn runs code generation, public-test execution, and self-check/handoff; repair code is rerun locally before the current code and relay packet move to the next specialist or final submission.
Table 1: Main results. Pass rate is solved-task fraction. Time in sec, tokens in thousands and number of calls are per-task. All values averaged across 3 runs. CodeSIM∗ is our rerun on Gemma4.
MethodPass rateSecTokensCalls
Direct0.48± 0.0234.9± 49.61.8± 1.21.0± 0.0
Single-RAG0.53± 0.0159.8± 21.225.0± 3.212.0± 0.0
Parallel ens.0.56± 0.00360.9± 172.929.8± 5.117.6± 0.8
Base relay0.55± 0.00191.6± 91.534.1± 10.217.1± 1.7
MARS0.62± 0.01244.3± 154.440.3± 8.116.6± 1.3
CodeSIM∗0.73± 0.01817.5± 1358.532.2± 54.810.2± 14.2
Figure 2: Team-selection frequency per specialist. Fraction of tasks on which each specialist joins the team, averaged over runs. Base relay and MARS share a deterministic assessor and coincide; the Parallel ensemble uses an earlier, less selective prompt and spreads selections more widely, which does not translate into accuracy.
Figure 2: Team-selection frequency per specialist. Fraction of tasks on which each specialist joins the team, averaged over runs. Base relay and MARS share a deterministic assessor and coincide; the Parallel ensemble uses an earlier, less selective prompt and spreads selections more widely, which does not translate into accuracy.
Table 2: Backbone and language transfer. The same 165 tasks and final harness as Table 1, using recorded model- and method-specific decoding. Time is in seconds, tokens in thousands, and calls per task; PairCoder includes embedding traffic.
MethodPass rateSecTokensCalls
Qwen3.5-27B, C++17
Direct0.192±0.01827.2±74.61.7±2.21.0±0.0
Single-RAG0.264±0.00471.0±57.126.5±4.012.0±0.0
MARS0.297±0.012109.7±99.740.5±18.715.3±3.3
GPT-5.4-mini, C++17
Direct0.149±0.0193.4±2.31.2±0.51.0±0.0
Single-RAG0.364±0.02449.7±26.324.7±3.212.0±0.1
MARS0.503±0.04392.2±61.136.8±7.415.8±1.8
Gemma 4, Python (PyPy 3)
Direct0.485±0.00070.0±137.81.6±1.01.0±0.0
MARS0.622±0.015307.6±215.742.9±9.117.1±1.4
PairCoder0.705±0.009426.7±451.727.6±20.97.6±5.4
Figure 3: Average pass rate by Codeforces difficulty tier. MARS dominates prompt-only baselines on Medium and more than doubles Direct on Hard; CodeSIM∗ leads on every tier. Tier sizes: Easy n=56, Medium n=39, Hard n=70.
Figure 3: Average pass rate by Codeforces difficulty tier. MARS dominates prompt-only baselines on Medium and more than doubles Direct on Hard; CodeSIM∗ leads on every tier. Tier sizes: Easy n=56, Medium n=39, Hard n=70.
Table 3: Ablations and protocol variants on Gemma 4. The generalist variant also disables RAG; the last two rows are the broader Base relay and Parallel ensemble comparisons from Table 1.
ConfigurationPass rateΔ
MARS (full)0.624±0.006—
w/o RAG grounding0.604±0.007−0.020
Generalists, no RAG0.615±0.013−0.009
Earlier Base relay0.552±0.000−0.072
Parallel manager0.564±0.000−0.060
Figure 8: Example MARS trace for Codeforces 1620_B with three contributing specialists. Step 1 includes a within-step self-check repair loop: the first draft fails the public sample, the same specialist consumes the test report and emits a corrected draft before handing off. Later steps show the specialist contribution, public-test outcome, and structured handoff to the next agent.
Figure 8: Example MARS trace for Codeforces 1620_B with three contributing specialists. Step 1 includes a within-step self-check repair loop: the first draft fails the public sample, the same specialist consumes the test report and emits a corrected draft before handing off. Later steps show the specialist contribution, public-test outcome, and structured handoff to the next agent.

研究结果

  • 在165道CodeContests测试题、Gemma 4骨干上,MARS平均每题2.3个记录阶段即达到0.624±0.006的通过率,比Direct高14.4个百分点,比Single-RAG高9.5个百分点,比Parallel ensemble高6.0个百分点。
  • CodeSIM通过率最高,为0.731±0.009,但MARS以3.3倍更低的实际耗时(每题约244.3秒,对比CodeSIM重跑约817秒)缩小了大部分差距。
  • 按难度分层看,Easy几乎接近满分(0.80-0.93),而MARS相对Direct的优势在Medium(0.72对0.59)和Hard(0.40对0.18)上进一步拉大。
  • 换用其他骨干模型(Qwen3.5-27B、GPT-5.4-mini)以及改用Python语言时,Direct < Single-RAG < MARS的排序依然保持;在Python上MARS达到0.622±0.015,比Direct的0.485高13.7个百分点。
  • 去掉检索增强(RAG)后通过率下降2.0个百分点,Base relay和Parallel manager分别比MARS低7.2和6.0个百分点。

可应用场景

  • 为竞赛编程类或高度依赖算法知识的编码任务设计流水线时,可参考用检索支撑的分主题专家小队来替代单一通才模型的思路。
  • 可用于设计让智能体每轮都用公开测试自查、并自主决定保留、修复或交棒的自检式工作流。
  • 在需要控制实际运行耗时和token花费波动、同时又想保留重型迭代搜索系统大部分准确率的实际代码生成服务中作为参考方案。

局限与待验证事项

  • 评测仅限于165道CodeContests题目、三种骨干模型、两种语言(C++17和Python)、一个算法理论语料库(cp-algorithms)以及Codeforces标签体系,尚未验证在此范围之外的泛化能力。
  • 确定性门控只能拦截同一轮内公开测试的回退,无法覆盖隐藏测试,也无法跨专家比较;所有生成代码仍需要沙箱执行验证。
  • 只有CodeSIM在完全相同的阶段化协议下被直接比较,PairCoder仅在Python上比较,LDB、LPW、MapCoder、MaintainCoder、Xolver等因语言或协议差异未能移植对比。
  • 从Base relay到MARS的提升(0.552到0.624)同时包含公开测试自查、子任务追踪、基础设施修复等多项改动,各项改动的独立贡献未被拆分。
  • 去掉RAG的通才对比同时改变了检索本身,因此并未纯粹分离出'专业化'这一因素单独的效果。

为什么重要

对于正确信号稀少、又高度依赖深层算法知识的竞赛编程任务来说,这项工作说明用检索为专家智能体注入真实主题知识,可以在远低于重型系统成本的情况下追平大部分性能差距。对于设计多智能体编程流水线的开发者而言,这提供了一种替代通用角色分工的轻量方案:按主题接力、每步都有执行反馈的专家团队。

本文术语

  • 检索增强生成(RAG) · 模型生成回答前先检索外部资料并参考其内容的技术
  • 接力式流水线(relay pipeline) · 一个智能体写出的代码被依次交给下一个智能体接手修改的流程结构
  • 通过率(pass rate) · 全部题目中被判定为正确解决的比例
  • ExecEval · 实际运行生成代码、检验其是否通过公开测试样例的沙箱执行环境
  • CodeSIM · 通过反复计划、模拟、调试来生成代码的对比多智能体系统

论文原文摘要（英文）

Large Language Models excel at code generation, yet competitive programming exposes a persistent failure mode: existing multi-agent pipelines distribute work over generic planner, coder, and debugger roles and delegate the choice of algorithmic technique to the backbone alone. We present MARS (Multi-Agent Relay of Specialized LLMs), a prompt-only framework in which each agent is a topic specialist---dynamic programming, graphs, strings, geometry, and so on---grounded by retrieval-augmented generation over an algorithm-theory corpus. Given a problem, retrieval selects a small team of relevant specialists; a starter writes an initial C++17 solution, and each subsequent turn runs the candidate against public examples in a sandbox, lets the active specialist keep, repair, or hand off the draft, and forwards a structured packet to the next specialist. A single infrastructure-fixer pass normalizes boilerplate at the end. On the CodeContests test split with Gemma 4, MARS reaches $0.624 \pm 0.006$ pass rate at $2.3$ recorded pipeline stages per task ($+14.4$ percentage points over direct prompting), closing most of the gap to CodeSIM ($0.731$) at $3.3{\times}$ lower wall-clock cost and substantially smaller variance in per-task token spend. The source code is available on GitHub: https://github.com/fckand/mars.

作者 · Andrei Mikhailov, Mikhail Burtsev, Alsu Sagirova

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源： Andrei Mikhailov et al., arXiv:2608.23918, CC BY 4.0