Research map / Large language models / Retrieval and knowledge
On scalable oversight with weak LLMs judging strong LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies debate, consultancy, and direct QA as scalable oversight protocols where weaker LLM judges evaluate stronger LLM debaters/consultants across extractive, closed, and multimodal tasks.
Problem
Whether debate between strong LLMs allows weaker LLM judges to accurately supervise them across a broad range of capability gaps, beyond information asymmetry on reading comprehension.
Approach
Evaluates debate, consultancy, and their open-role variants on nine tasks spanning extractive, closed, and multimodal QA, varying judge strength relative to debaters, in an inference-only setting, and measures persuasiveness via Elo.
Key results
- Debate consistently outperforms consultancy across all tasks in judge accuracy.
- Debate beats QA without article for GPT-3.5 on closed tasks (p=0.037), but not for Pro 1.0 (p=0.1) or Pro 1.5; QA with article is best on extractive tasks.
- In open consultancy, judges are similarly convinced whether the consultant is correct or not, while open debate gives much higher judge accuracy when the protagonist is incorrect.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use debate rather than consultancy | LLM oversight protocols · Extractive QA and closed QA | judge accuracy | Helped |
| use debate rather than consultancy | LLM oversight protocols · Multimodal QA | judge accuracy | Helped |
| use debate rather than QA without article | LLM judges · Extractive QA | judge accuracy | Helped |
| use debate rather than QA with article | LLM judges · Extractive QA | judge accuracy | Hurt |
| use debate rather than QA without article | LLM judges · Closed QA | judge accuracy | Mixed |
| use debate rather than QA without article | LLM judges · Multimodal QA | judge accuracy | No clear effect |
| reduce the number of turns from 3 to 1 (ablation) | consultancy and debate · Evaluated tasks | judge accuracy: 3 to 1 | No clear effect |
| reduce debater Best-of-N sampling from 4 to 1 (ablation) | debate · Evaluated tasks | judge accuracy: 4 to 1 | No clear effect |
| use 6-shot rather than 0-shot judge prompting (ablation) | LLM judges · Evaluated tasks | judge accuracy: 0-shot to 6-shot | No clear effect |
| add chain-of-thought reasoning for the judge (ablation) | LLM judges · Evaluated tasks | judge accuracy | Mixed |
| use sequential rather than simultaneous debate turns (ablation) | debate · Evaluated tasks | judge accuracy | No clear effect |
| evaluate with both answer and debate orders swapped (ablation) | debate · Evaluated tasks | mean judge accuracy | No clear effect |
Surprising findings
- Chain-of-thought for the judge tends to harm or not help judge accuracy.
- Best-of-N sampling, number of turns, and few-shot prompting have little effect on judge accuracy.
Limitations
- Inference-only setting; no training of debaters or judges via the protocol.
- Models are RLHF-tuned for honesty, which may hamper dishonest debaters.
- Uses AI judges rather than humans and models far from superhuman.
Experimental setup
Datasets: QuALITY BoolQ GPQA MMLU GSM8K PrOntoQA TruthfulQA MMMU
Benchmarks: QuALITY BoolQ GPQA-extractive MMLU GSM8KQA PrOntoQA TruthfulQA GPQA MMMU
Metrics: judge accuracy win rate Elo
Baselines: consultancy QA without article QA with article open consultancy
Builds on: debate (Irving et al.) consultancy QuALITY debate experiments (Khan et al.) chain-of-thought prompting
Closest papers by meaning
- Collaborative Disagreement Resolution for Scalable Oversight (ICML 2026 · similarity 0.91)
- When Debate Helps: Proposal Supply and Verification-Aware Readout in Multi-Agent Reasoning (NeurIPS 2026 · similarity 0.89)
- Debating with More Persuasive LLMs Leads to More Truthful Answers (ICML 2024 · similarity 0.89)
- Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs (ICML 2024 · similarity 0.89)
- Multi-Agent Debate for LLM Judges with Adaptive Stability Detection (NeurIPS 2025 · similarity 0.88)
- Multi-LLM Debate: Framework, Principals, and Interventions (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.