Adversarial Review: Structured Disagreement for Grounded Agentic Code Review

arXiv:2608.181672026-08-20

For AI code review, one reviewer plus one critic beats piling on more agents

Researchers found that adding more AI agents to a coding team gives shrinking returns, so instead they built a lean protocol called Adversarial Review (AR) where just one reviewer agent and one critic agent argue over a code review before the main coding agent edits anything. AR beat a five-agent baseline on a coding benchmark, but on a real pull-request review benchmark the two agents initially just agreed with each other too easily, a problem the authors fixed by forcing the critic to always cite concrete code evidence when disagreeing. The fixed version then topped that benchmark too, and AR also outperformed baselines on a large real-world bug-fixing benchmark, though at higher computational cost.

What they did

On LiveCodeBench (coding problems), AR — a main agent plus just one reviewer and one critic — reached an 87% pass rate versus 85% for MARS, a rival method that uses five agents (three reviewers plus a meta-reviewer)
On SWE-PRBench, a benchmark of real GitHub pull-request reviews, plain AR actually scored lowest of the methods tested (F1 = 0.457) because the reviewer and critic tended to agree with each other without enough evidence, a failure the authors call 'false consensus'
Changing only the critic's prompt — from a simple agree/disagree choice to three options (agree, disagree with cited code evidence, or disagree with an unverified concern) — raised AR's score to F1 = 0.533, the best in that test
On SWE-bench Verified, which asks agents to fix 500 real GitHub issues, AR resolved 75.2% of tasks versus 71.6% for a no-collaboration baseline and 72.6% for MARS, though AR used about 4.5 times more computing tokens than the baseline
The design keeps code edits separate from review discussion: the reviewer and critic can only exchange review text, and the main agent only edits the code after the two have reached a stable, evidence-backed agreement

Table 1: LCB results. The leader is bold. The first four methods cluster at the same pass-rate. MARS breaks out of the cluster. AR scores highest with the fewest agents among the methods that broke out.
Method	pass / 105	pass-on-hard / 57	# agents
Zero-shot	77%	35/57 (61%)	1
Self-Refine	77%	35/57 (61%)	1
Single-reviewer	77%	36/57 (63%)	2
Two-reviewers	75%	34/57 (60%)	3
MARS	82%	39/57 (68%)	5
AR	87%	43/57 (75%)	3

Table 2: SWE-PRBench review-relevant subset. The leader is bold. “AR” and “AR with text constraint” share the same R+C structure; only the prompts change. The number of agents and the way they connect did not change. The prompt iteration is described in Section 4.4.
Method	F1	N
AR with text constraint	0.533	100
Two-reviewers	0.503	100
MARS	0.501	100
Single-reviewer	0.495	100
AR	0.457	100

Table 3: SWE-bench Verified pass-rate of three methods: Zero-shot, MARS, and AR.
Method	pass-rate (%)	N
AR	75.2%	500
Zero-shot	71.6%	500
MARS	72.6%	500

Why it matters

As companies increasingly deploy AI agents to write and review code automatically, this work shows that simply adding more agents to the team is not the answer — a small, tightly structured disagreement between two agents works better and cheaper. It also exposes a concrete risk for anyone relying on multi-agent AI review: agents can rubber-stamp each other's mistakes, so systems need to be explicitly designed to force evidence-based pushback rather than easy agreement.

Terms in this paper

LLM agent · a program built on a large language model that plans and carries out tasks, sometimes using tools
subagent · a helper AI agent that the main agent calls for a specific subtask, similar to using a tool
multi-agent debate (MAD) · a setup where several AI agents exchange arguments back and forth to refine an answer
pass rate · the percentage of tasks the AI solves correctly or that pass automated tests
F1 score · a metric combining precision and recall, used here to measure how well AI review comments match real human reviewer comments

Figures we cannot republish

Figure 1: Workflow of Adversarial Review (AR). The main agent first produces artifact version N (code or plan). The protocol then enters an inner loop in which the artifact is frozen: reviewer R generates Reviewk, critic C evaluates and may revise that review, and R responds until a consistent review is reached. If the review converges on the first pass and identifies no flaws, the artifact is accepted immediately. Otherwise, the main agent edits the artifact to produce version N+1, updates the change log, and the process repeats. The key separation is that the inner loop exchanges review text only, while artifact edits occur only in the outer loop.

See the figures in the original paper →

Original abstract (English)

Early multi-agent LLM systems often used role-separated teams, yet scaling agent count yields diminishing returns on repository-level coding tasks. Recent alternatives treat agents as passive tools (subagents), yet this removes the benefits of agent interaction entirely. We study whether a subagent paradigm can support a middle ground: minimal agentic cooperation without the overhead of large multi-agent teams. We introduce Adversarial Review (AR),

Authors · Eric S. Qiu, Joyce Gill

Read on arXiv

Latest papers

All papers →