Research map / Large language models / Language, safety and interpretability
The Consensus Game: Language Model Generation via Equilibrium Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes equilibrium-ranking, a training-free game-theoretic LM decoding method that reconciles generative and discriminative scoring by computing regularized equilibria of a signaling game solved with piKL no-regret dynamics.
Problem
Language models give inconsistent answers when queried generatively (sampling or ranking by likelihood) versus discriminatively (scoring candidate correctness), and existing ensembling or deliberation fixes are poorly calibrated or costly.
Approach
Casts LM decoding as a regularized imperfect-information signaling game between a generator and discriminator, initialized from LM-derived policies (with a false-prompt contrast), and computes approximate regularized equilibria via piKL no-regret dynamics to obtain consensus predictions without training.
Key results
- LLaMA-7B with ER-D reaches 39.9 on zero-shot MMLU, above 5-shot LLaMA-7B (35.1) and zero-shot GPT3-175B (37.7).
- LLaMA-13B with ER-D reaches 61.4 on ARC-Challenge, above LLaMA-65B (56.0) and PaLM-540B (53.0).
- On TruthfulQA, LLaMA-13B with ER-G reaches 39.83 BLEU-Acc versus 33.05 for greedy decoding.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ER-D instead of discriminative ranking | LLaMA-7B · MMLU | accuracy: 40.4 -> 39.9 | No clear effect |
| Apply ER-D instead of discriminative ranking | LLaMA-13B · MMLU | accuracy: 41.9 -> 45.1 | Helped |
| Apply ER-D instead of generative ranking | LLaMA-7B · ARC-Easy | accuracy: 68.2 -> 71.5 | Helped |
| Apply ER-D instead of self-contrastive ranking | LLaMA-13B · ARC-Challenge | accuracy: 59.3 -> 61.4 | Helped |
| Apply ER-D instead of self-contrastive ranking | LLaMA-13B · RACE-Middle | accuracy: 64.8 -> 68.6 | Helped |
| Apply ER-D instead of self-contrastive ranking | LLaMA-7B · RACE-High | accuracy: 53.1 -> 56.4 | Helped |
| Apply ER-D instead of discriminative ranking | LLaMA-7B · HHH | accuracy: 70.1 -> 71.5 | Helped |
| Apply default ER-D instead of discriminative ranking | LLaMA-13B · HHH | accuracy: 69.2 -> 61.1 | Hurt |
| Use ER-G instead of self-contrastive ranking | LLaMA-7B · TruthfulQA | BLEU-Acc: 34.91 ± 0.57 -> 34.61 ± 0.99 | Hurt |
| Use ER-G instead of discriminative ranking | LLaMA-13B · TruthfulQA | BLEU-Acc: 39.05 ± 1.42 -> 39.83 ± 2.20 | Helped |
| Use ER-D instead of majority vote | LLaMA-7B · GSM8K | accuracy: 14.7 ± 0.2 -> 15.1 ± 0.6 | No clear effect |
| Use ER-D instead of majority vote | LLaMA-13B · GSM8K | accuracy: 22.5 ± 0.5 -> 23.0 ± 0.5 | No clear effect |
Surprising findings
- Equilibrium ranking on LLaMA-7B outperforms the much larger LLaMA-65B and PaLM-540B on several benchmarks.
- On TruthfulQA, the LLaMA-13B gains are larger than LLaMA-7B, suggesting the method mitigates negative scaling on this benchmark.
Limitations
- Candidate set is finite and built from sampling, so correct answers outside the candidates cannot be selected.
- Convergence of last iterate to a regularized Nash equilibrium is not proven in this setting.
- Gains are mixed on some tasks, e.g. ARC-Easy and HHH where discriminative ranking or the generative baseline is competitive.
Experimental setup
Datasets: MMLU ARC RACE HHH TruthfulQA GSM8K
Benchmarks: MMLU ARC-Easy ARC-Challenge RACE-middle RACE-high HHH TruthfulQA GSM8K
Metrics: accuracy BLEU-Acc
Baselines: Generative Ranking Mutual Information Ranking Self-Contrastive Ranking Discriminative Ranking Greedy decoding Majority Vote LLaMA-65B PaLM-540B
Builds on: piKL counterfactual regret minimization contrastive decoding self-consistency
Closest papers by meaning
- Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategies (ICLR 2025 · similarity 0.90)
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.89)
- Learning How Hard to Think: Input-Adaptive Allocation of LM Computation (ICLR 2025 · similarity 0.88)
- Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · similarity 0.87)
- Incentivizing Truthful Language Models via Peer Elicitation Games (NeurIPS 2025 · similarity 0.87)
- Improving Open-Ended Text Generation via Adaptive Decoding (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.