Research map / Large language models / Language, safety and interpretability
Ask Me Anything: A simple strategy for prompting language models
Simran Arora, Avanika Narayan, Mayee F Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, Christopher Re
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AMA question-generation and answer-generation prompt chains instead of few-shot prompting | GPT-J-6B · 20 language benchmarks | average performance: 41% | Improved |
| Reformat restrictive prompts as open-ended question-answering prompts | small model on CB, RTE, and WSC · CB, RTE, WSC | average performance: 41.7% to 71.5% (+72%) | Improved |
| Ask precise questions based on the pronoun's local context | GPT-J-6B · WSC | accuracy: 69.2% to 74.7%; 49.4% lift | Improved |
| Add a mapping step from open-ended answer to a valid class | open-ended classification prompting · DBPedia | accuracy: 33.3% lift | Improved |
| Use weak supervision rather than majority vote to aggregate prompt outputs | AMA prompt chains · 20 tasks | accuracy: up to 8.7 points; worse on 4 tasks by at most 1.0 points | Mixed |
| Model dependencies among prompt chains rather than assuming conditional independence (ablation) | AMA aggregation · 9 tasks | accuracy: up to 9.6 points; average 2.2 points | Improved |
| Remove question-answering reformatting and aggregate few-shot prompts only (ablation) | GPT-J-6B few-shot prompting · CB | accuracy: 50.0 to 83.9 | Hurt |
| Remove question-answering reformatting and aggregate few-shot prompts only (ablation) | GPT-J-6B few-shot prompting · RTE | accuracy: 54.2 to 75.1 | Hurt |
| Remove question-answering reformatting and aggregate few-shot prompts only (ablation) | GPT-J-6B few-shot prompting · WSC | accuracy: 38.5 to 77.9 | Hurt |
| Use weak supervision instead of majority vote over PromptSource formats (ablation) | T0 (3B) · RTE | accuracy: 60.6 to 69.7 | Improved |
| Use weak supervision rather than majority vote on AMA prompt chains (ablation) | GPT-J-6B · RTE | accuracy: 66.0 to 75.1 | Improved |
| Use weak supervision rather than majority vote on AMA prompt chains (ablation) | GPT-J-6B · StoryCloze | accuracy: 87.9 to 87.8 | No clear effect |
Closest papers by meaning
- Prompt Sketching for Large Language Models (ICML 2024 · similarity 0.91)
- Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.91)
- On the Worst Prompt Performance of Large Language Models (NeurIPS 2024 · similarity 0.90)
- Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective (ICML 2025 · similarity 0.89)
- Rethinking Generative Large Language Model Evaluation for Semantic Comprehension (ICML 2024 · similarity 0.89)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
