Research map / Large language models / Instruction and fine-tuning
Incentivizing Truthful Language Models via Peer Elicitation Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Peer Elicitation Games (PEG), a training-free game-theoretic framework where multiple LLM discriminators are rewarded via a determinant-based mutual information peer-prediction payment, with a generator aligned to their majority vote, and reports truthfulness, no-regret, and last-iterate convergence results plus accuracy gains on ARC, MMLU, and GPQA.
Problem
LLMs produce inconsistent and untruthful outputs, and existing alignment methods need supervision, fine-tuning, or rely on agreement-based objectives (consensus games) that can reward collusion rather than truthfulness.
Approach
Discriminators from distinct base LLMs are paid by a determinant-based mutual information score over pairwise co-report matrices (peer prediction style), making truthful reporting a dominant strategy without ground truth; policies are updated via online mirror descent and the generator is trained to match the discriminators' majority vote.
Key results
- On ARC-Challenge, PEG reaches 87.01% accuracy vs 78.03% for the best baseline (MI) and 70.68% for discriminative ranking.
- On MMLU, PEG reaches 70.73% vs 59.98% for generative ranking.
- With 3 discriminators PEG reaches 87.01% on ARC-Challenge; with 7 discriminators the gain shrinks to 81.97% vs 81.54% for ER-D.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PEG instead of the strongest baseline (MI) | Qwen2.5-7B-Instruct · ARC-Easy | accuracy: 88.61 -> 91.78 | Helped |
| Apply PEG instead of the strongest baseline (MI) | Qwen2.5-7B-Instruct · ARC-Challenge | accuracy: 78.03 -> 87.01 | Helped |
| Apply PEG instead of the strongest baseline (G) | Qwen2.5-7B-Instruct · MMLU | accuracy: 59.98 -> 70.73 | Helped |
| Apply PEG instead of the strongest baseline (G) | Qwen2.5-7B-Instruct · GPQA-Main | accuracy: 18.08 -> 22.54 | Helped |
| Use 5 rather than 3 discriminators (ablation) | PEG · ARC-Challenge | accuracy: 87.01 -> 86.75 | No clear effect |
| Use 7 rather than 5 discriminators (ablation) | PEG · ARC-Challenge | accuracy: 86.75 -> 81.97 | Hurt |
Surprising findings
- Accuracy slightly decreases when expanding from 5 to 7 discriminators, attributed to violation of conditional independence among similar discriminators.
- The weakest discriminator (OQwen-7B at start) becomes the most accurate after PEG on some benchmarks, though the 7-model setting shows only 81.97% majority accuracy.
Limitations
- Theoretical assumptions (conditional independence, local concavity, gradient boundedness) are strong and only mildly validated; performance degrades with more similar discriminators.
- Evaluated only on multiple-choice QA benchmarks with 7B-class models; no evaluation on open-ended generation.
- Experiments use a 'training-free' framing yet the method is described as updating policies via OMD and the generator, leaving the actual update procedure ambiguous.
Experimental setup
Datasets: ARC-Easy ARC-Challenge MMLU GPQA
Benchmarks: ARC-Easy ARC-Challenge MMLU GPQA-Main
Metrics: accuracy
Baselines: Generative Ranking Discriminative Ranking Mutual Information Ranking ER-D (Consensus Game)
Builds on: Consensus Game peer prediction online mirror descent multi-task peer prediction
Closest papers by meaning
- Carrot and Stick: Eliciting Comparison Data and Beyond (NeurIPS 2024 · similarity 0.88)
- LLM Active Alignment: A Nash Equilibrium Perspective (NeurIPS 2026 · similarity 0.88)
- LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models (AAAI 2026 · similarity 0.87)
- Improving Cooperation in Language Games with Bayesian Inference and the Cognitive Hierarchy (AAAI 2025 · similarity 0.87)
- The Consensus Game: Language Model Generation via Equilibrium Search (ICLR 2024 · similarity 0.87)
- Evaluating Language Model Agency Through Negotiations (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.