Adversarial Review: Structured Disagreement for Grounded Agentic Code Review
AI代码审查:与其堆更多智能体,不如让一个审查者和一个批评者互相较真
研究者发现,给AI编程系统堆更多智能体带来的收益会逐渐减少,于是他们设计了一种精简协议Adversarial Review(AR),只用一个审查者智能体和一个批评者智能体互相争论,主编程智能体等两者意见稳定后才修改代码。AR在代码生成基准上击败了用五个智能体的对手方法,但在真实拉取请求审查基准上,两个智能体一开始过于容易达成没有依据的一致,后来通过强制批评者引用具体代码证据来反驳才解决了这个问题,改进后的版本在该基准上也拿到最高分,在真实修复Bug的大型基准上同样优于基线,不过计算成本更高。
他们做了什么
- 在LiveCodeBench编程题基准上,只用一个主智能体加一个审查者、一个批评者的AR通过率达到87%,高于使用五个智能体（三个审查者加一个元审查者）的对手方法MARS的85%
- 在真实GitHub拉取请求审查基准SWE-PRBench上,朴素版AR的得分反而是所测方法中最低的（F1为0.457）,原因是审查者和批评者常常在缺乏证据的情况下就相互认同,作者称之为“虚假共识”
- 仅修改批评者的提示词——把简单的“同意/不同意”改成三种选项（同意、引用具体代码证据反驳、提出无证据支持的疑虑）——就把AR的F1提升到0.533,成为该测试中的最高分
- 在要求修复500个真实GitHub问题的SWE-bench Verified基准上,AR的解决率达到75.2%,高于无协作基线的71.6%和MARS的72.6%,但AR消耗的计算token约为基线的4.5倍
- 该设计把代码修改和审查讨论严格分开:审查者和批评者之间只交换审查文字,主智能体只有在两者就审查结果达成稳定、有证据支撑的一致后才会真正修改代码
| Method | pass / 105 | pass-on-hard / 57 | # agents |
|---|---|---|---|
| Zero-shot | 77% | 35/57 (61%) | 1 |
| Self-Refine | 77% | 35/57 (61%) | 1 |
| Single-reviewer | 77% | 36/57 (63%) | 2 |
| Two-reviewers | 75% | 34/57 (60%) | 3 |
| MARS | 82% | 39/57 (68%) | 5 |
| AR | 87% | 43/57 (75%) | 3 |
| Method | F1 | N |
|---|---|---|
| AR with text constraint | 0.533 | 100 |
| Two-reviewers | 0.503 | 100 |
| MARS | 0.501 | 100 |
| Single-reviewer | 0.495 | 100 |
| AR | 0.457 | 100 |
| Method | pass-rate (%) | N |
|---|---|---|
| AR | 75.2% | 500 |
| Zero-shot | 71.6% | 500 |
| MARS | 72.6% | 500 |
为什么重要
随着企业越来越多地部署AI智能体自动编写和审查代码,这项研究表明单纯增加智能体数量并非答案,一种小而结构化的“对抗式”分歧机制反而更有效也更省成本。它也揭示了依赖多智能体AI审查的一个具体风险:智能体之间可能互相盲目附和彼此的错误,因此系统必须被明确设计成强制基于证据的反驳,而不是轻易达成一致。
本文术语
- LLM智能体 · 基于大语言模型构建、能够规划并执行任务、有时会调用工具的程序
- 子智能体 · 主智能体为完成特定子任务而调用的辅助AI智能体,使用方式类似工具
- 多智能体辩论(MAD) · 多个AI智能体互相交换论点以完善答案的机制
- 通过率 · AI正确解决任务或通过自动化测试的比例
- F1分数 · 综合精确率和召回率的评价指标,这里用来衡量AI审查评论与真实人类评审意见的匹配程度
无法转载的图表
- Figure 1: Workflow of Adversarial Review (AR). The main agent first produces artifact version N (code or plan). The protocol then enters an inner loop in which the artifact is frozen: reviewer R generates Reviewk, critic C evaluates and may revise that review, and R responds until a consistent review is reached. If the review converges on the first pass and identifies no flaws, the artifact is accepted immediately. Otherwise, the main agent edits the artifact to produce version N+1, updates the change log, and the process repeats. The key separation is that the inner loop exchanges review text only, while artifact edits occur only in the outer loop.
论文原文摘要（英文）
Early multi-agent LLM systems often used role-separated teams, yet scaling agent count yields diminishing returns on repository-level coding tasks. Recent alternatives treat agents as passive tools (subagents), yet this removes the benefits of agent interaction entirely. We study whether a subagent paradigm can support a middle ground: minimal agentic cooperation without the overhead of large multi-agent teams. We introduce Adversarial Review (AR),
在 arXiv 阅读最新论文
- FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI SystemsAI经常能说对财务对账出错的原因,却拿不出真正的证据
- Looped Language Models Improve Compositional Tool Calling会反复回想自己答案的AI模型,更擅长按顺序组合调用多个工具
- FACET: Preserving Source Intent and Executable State in Terminal Task SynthesisFACET:让终端命令行任务的“说明书、环境、答案、判分器”自动保持一致
- FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents让AI连续管理一家足球俱乐部20年后发现,胜负关键不在模型大小,而在经营习惯
- Bridging Search and CRM: Productionizing AI Product Research Agents for Customer Re-EngagementAI智能体追着离场用户发WhatsApp,把逛而不买的顾客拉回来
- GenEx: A Graph-Based Representational Paradigm for SARS-CoV-2 Variant Detection via Codon Co-occurrence Networks把病毒基因序列变成密码子关系网络图,用来区分新冠变异株
- Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models语言模型全程冻结,只训练一个小连接器,也能做出好用的听觉理解AI
- Which Negatives Matter? Ask Your Text Encoder: Adaptive Similarity Margins for Dense-Caption Retrieval图文检索AI误以为自己已经全学会了,结果学不会区分那些几乎一样的描述句子