AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Ask Me Anything: A simple strategy for prompting language models

Simran Arora, Avanika Narayan, Mayee F Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, Christopher Re

ICLR 2023 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 75 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use AMA question-generation and answer-generation prompt chains instead of few-shot promptingGPT-J-6B · 20 language benchmarksaverage performance: 41%Improved
Reformat restrictive prompts as open-ended question-answering promptssmall model on CB, RTE, and WSC · CB, RTE, WSCaverage performance: 41.7% to 71.5% (+72%)Improved
Ask precise questions based on the pronoun's local contextGPT-J-6B · WSCaccuracy: 69.2% to 74.7%; 49.4% liftImproved
Add a mapping step from open-ended answer to a valid classopen-ended classification prompting · DBPediaaccuracy: 33.3% liftImproved
Use weak supervision rather than majority vote to aggregate prompt outputsAMA prompt chains · 20 tasksaccuracy: up to 8.7 points; worse on 4 tasks by at most 1.0 pointsMixed
Model dependencies among prompt chains rather than assuming conditional independence (ablation)AMA aggregation · 9 tasksaccuracy: up to 9.6 points; average 2.2 pointsImproved
Remove question-answering reformatting and aggregate few-shot prompts only (ablation)GPT-J-6B few-shot prompting · CBaccuracy: 50.0 to 83.9Hurt
Remove question-answering reformatting and aggregate few-shot prompts only (ablation)GPT-J-6B few-shot prompting · RTEaccuracy: 54.2 to 75.1Hurt
Remove question-answering reformatting and aggregate few-shot prompts only (ablation)GPT-J-6B few-shot prompting · WSCaccuracy: 38.5 to 77.9Hurt
Use weak supervision instead of majority vote over PromptSource formats (ablation)T0 (3B) · RTEaccuracy: 60.6 to 69.7Improved
Use weak supervision rather than majority vote on AMA prompt chains (ablation)GPT-J-6B · RTEaccuracy: 66.0 to 75.1Improved
Use weak supervision rather than majority vote on AMA prompt chains (ablation)GPT-J-6B · StoryClozeaccuracy: 87.9 to 87.8No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy