Research map / Large language models / Instruction and fine-tuning
Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information
Rui Ai, Yuqi Pan, David Simchi-Levi, Milind Tambe, Haifeng Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ISP instead of majority voting | LLM aggregation · Simulated (K=4) | accuracy: 92.64% -> 94.45% | Improved |
| use ISP instead of majority voting | LLM aggregation · Simulated (K=2) | accuracy: 85.13% -> 90.48% | Improved |
| use ISP instead of majority voting | LLM aggregation · Simulated (K=10) | accuracy: 95.54% -> 96.49% | Improved |
| use surprisingly popular instead of majority voting | LLM aggregation · Simulated (K=2) | accuracy: 85.13% -> 79.94% | Hurt |
| use OW-L instead of majority voting | four strong LLMs · UltraFeedback | accuracy: 72.21% -> 73.66% | Improved |
| use OW-L instead of majority voting | four strong LLMs · MMLU | accuracy: 89.32% -> 90.37% | Improved |
| use OW-L instead of majority voting | four strong LLMs · ARMMAN | accuracy: 85.24% -> 85.78% | Improved |
| aggregate strong models with OW instead of picking oracle single best | four strong LLMs · MMLU | accuracy: 91.02% -> 90.37% | Hurt |
| use OW-L instead of majority voting on hard-distractor subset | four strong LLMs · MMLU-hard | accuracy: 17.23% -> 24.79% | Improved |
| use ISP instead of majority voting on hard-distractor subset | four strong LLMs · MMLU-hard | accuracy: 17.23% -> 18.66% | Improved |
| ensemble all eight models with OW-I instead of MV (ablation) | eight LLMs · UltraFeedback | accuracy: 70.23% -> 72.44% | Improved |
| ensemble all eight models with OW-L instead of MV (ablation) | eight LLMs · ARMMAN | accuracy: 84.40% -> 85.10% | Improved |
Closest papers by meaning
- Are More LLM Calls All You Need? Towards the Scaling Properties of Compound AI Systems (NeurIPS 2024 · similarity 0.88)
- Multi-Agent Debate for LLM Judges with Adaptive Stability Detection (NeurIPS 2025 · similarity 0.88)
- Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? (NeurIPS 2025 · similarity 0.87)
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · similarity 0.87)
- Investigating Non-Transitivity in LLM-as-a-Judge (ICML 2025 · similarity 0.86)
- LLM Active Alignment: A Nash Equilibrium Perspective (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
