Research map / Large language models / Language, safety and interpretability
Incentive-Aligned LLM Summaries
Yanchen Jiang, Zhe Feng, Aranyak Mehta
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace majority-claims re-summary with peer-prediction source filtering (TTS) | LLM summarization pipeline · NQ | answer accuracy: 38.6% -> 72.3% | Improved |
| replace majority-claims re-summary with peer-prediction source filtering (TTS) | LLM summarization pipeline · ClashEval | answer accuracy: 38.4% -> 77.1% | Improved |
| replace initial summary with TTS | LLM summarization pipeline · NQ | precision: 40.8% -> 76.1% | Improved |
| replace initial summary with TTS | LLM summarization pipeline · ClashEval | precision: 49.3% -> 86.2% | Improved |
| replace initial summary with TTS | LLM summarization pipeline · NQ | F1: 31.9% -> 50.0% | Improved |
| replace initial summary with TTS | LLM summarization pipeline · ClashEval | F1: 30.4% -> 56.1% | Improved |
| replace initial summary with TTS | LLM summarization pipeline · NQ | recall: 26.2% -> 37.2% | Improved |
| replace initial summary with TTS | LLM summarization pipeline · ClashEval | recall: 22.0% -> 31.5% | Improved |
| prompt LLM to use majority claims only instead of plain summary | LLM summarization pipeline · ClashEval | answer accuracy: 15.6% -> 30.2% | Improved |
| prompt LLM to use majority claims only instead of plain summary | LLM summarization pipeline · NQ | answer accuracy: 25.1% -> 27.5% | Improved |
| majority-claims filtering lowers recall relative to initial summary | Initial Synthesis · ClashEval | recall: 22.0% -> 20.4% | Hurt |
| use output-agreement majority scoring instead of multi-task peer prediction with 4 uninformative sources (ablation) | source scoring · ClashEval | avg score of adversarial source | Hurt |
Closest papers by meaning
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.86)
- Aligning Large Language Models for Faithful Integrity Against Opposing Argument (AAAI 2025 · similarity 0.86)
- Alignment for Honesty (NeurIPS 2024 · similarity 0.86)
- Information Discernment in Large Language Models (NeurIPS 2026 · similarity 0.86)
- Calibrating LLMs with Semantic-level Reward (NeurIPS 2026 · similarity 0.86)
- ReMoDetect: Reward Models Recognize Aligned LLM's Generations (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
