Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate¶
Wynn et al. (2025) challenge the prevailing assumption that multi-agent debate reliably improves LLM reasoning. Through systematic experiments across three tasks (CommonSenseQA, MMLU, GSM8K) and three models (GPT-4o-mini, LLaMA-3.1-8B, Mistral-7B), they demonstrate that debate can actively degrade group performance — particularly in heterogeneous teams where weaker models participate alongside stronger ones.
Key Findings¶
Debate can harm performance. On CommonSenseQA, debate consistently reduced accuracy across all agent configurations. On MMLU and GSM8K, many heterogeneous configurations also showed degradation after debate. This directly contradicts the narrative that more deliberation among AI agents always leads to better outcomes.
Performance degrades over debate rounds. In many configurations, accuracy declined as debate continued — more rounds of discussion made the group answer worse, not better. The social dynamics of debate can amplify errors rather than correcting them.
Correct-to-incorrect flips dominate. Analysis of answer transitions revealed that agents more frequently changed from correct answers to incorrect ones than the reverse. Debate causes agents to abandon initially correct positions more often than it causes them to correct initially wrong ones. This is the most concerning finding: debate is net-destructive for accuracy in these conditions.
Social pressure overrides reasoning quality. Agents in the minority position — even when they hold the correct answer — are significantly more likely to flip their answer to match the majority. The probability of an undesirable correct→incorrect flip is highest when zero other agents agree (the agent is isolated), and decreases as more agents share its position. This mirrors human conformity effects but without the compensating human ability to recognise and resist social pressure.
Weak agents drag down strong ones. Introducing a weaker model into a debate with stronger models can degrade the group's performance below what the stronger model achieves alone. The weaker model's incorrect but persuasive reasoning can mislead stronger agents — a form of inter-agent contamination.
Sycophancy is a contributing factor but not the full explanation. The authors tested a "correctness payoff" prompt intervention that explicitly instructed agents to prioritise accuracy over agreement. This intervention did not significantly reduce correct→incorrect flips, suggesting that the failure is not purely sycophantic but involves deeper mechanisms: sequential revision effects (conditioning on previous rounds), social conformity dynamics, and task-dependent reasoning fragility.
No single mechanism explains debate failure. The interplay of model capability, task characteristics, and social influence jointly determines debate outcomes. Sycophancy alone, social conditioning alone, or sequential revision alone cannot fully account for the observed degradation patterns.
Relevance to This Wiki¶
This paper provides critical evidence for multi-agent-coordination-failures — demonstrating that a common multi-agent pattern (debate/deliberation) can be actively harmful under specific conditions. Combined with Wu et al. (2025), the picture that emerges is nuanced: debate helps when participants are strong and diverse, but harms when capability gaps exist or on certain task types.
For safety-critical system design, the most important implication is that multi-agent deliberation cannot be assumed beneficial. System designers must empirically validate that their specific debate configuration improves outcomes on their specific task type, rather than citing the general multi-agent debate literature as justification. The finding that weak agents contaminate strong ones is particularly relevant: in safety-critical applications, introducing a cheaper or less capable model into a verification pipeline to reduce cost could degrade overall system reliability below single-strong-model performance. This connects to epistemic-independence — genuine independence requires not just separate models but models of sufficient capability to maintain their positions under social pressure.