Research map / Large language models / Retrieval and knowledge
Leveraging Large Language Models for Multiple Choice Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace cloze prompting with multiple choice prompting | Codex (Davinci), across 20 MCQA datasets · Zero-shot MCQA | accuracy: +8.3 percentage points on average | Helped |
| Replace cloze prompting with multiple choice prompting | Codex (Davinci), across 20 MCQA datasets · One-shot MCQA | accuracy: +12.2 percentage points on average | Helped |
| Replace cloze prompting with multiple choice prompting | Codex (Davinci), across 20 MCQA datasets · Few-shot MCQA | accuracy: +9.7 percentage points on average | Helped |
| Replace cloze prompting with multiple choice prompting | Codex (Davinci) · Cosmos QA, few-shot | accuracy: 38.1 -> 82.4; +44.3 percentage points | Helped |
| Replace cloze prompting with multiple choice prompting | Codex (Davinci) · PIQA, few-shot | accuracy: 86.1 -> 84.5 | Hurt |
| Replace cloze prompting with multiple choice prompting | Codex (Davinci) · RiddleSense, few-shot | accuracy: 91.3 -> 83.9 | Hurt |
| Replace cloze prompting with multiple choice prompting | Codex (Davinci) · Winogrande (XL), few-shot | accuracy: 75.5 -> 72.1 | Hurt |
| Replace cloze prompting with multiple choice prompting | Codex (Davinci) · AG News, few-shot | accuracy: 90.1 -> 89.4 | Hurt |
| Use MCP instead of cloze prompting (ablation) | Codex (Davinci) · Answer choices with random casing or inserted spaces | accuracy: Performance drops are 1.3% and 0.5% for MCP, versus 12.4% and 10.3% for SC | Helped |
| Strongly shuffle the answer-option ordering (ablation) | Codex + MCP · Winogrande (XS), few-shot | accuracy: 73.6 -> 66.1 | Hurt |
Closest papers by meaning
- Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions (ICLR 2025 · similarity 0.93)
- Rethinking Generative Large Language Model Evaluation for Semantic Comprehension (ICML 2024 · similarity 0.91)
- Listening to the Wise Few: Query–Key Alignment Unlocks Latent Correct Answers in Large Language Models (NeurIPS 2026 · similarity 0.90)
- Large Language Models Are Not Robust Multiple Choice Selectors (ICLR 2024 · similarity 0.89)
- SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs (ICLR 2024 · similarity 0.88)
- Cost-efficient Knowledge-based Question Answering with Large Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.