Research map / Large language models / AI and society
Prompting as Scientific Inquiry
Ari Holtzman, Chenhao Tan
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A position paper arguing that prompting, when run as structured behavioral experiments ("prompt science"), is a legitimate scientific method complementary to mechanistic interpretability, and rebutting common critiques of prompting.
Problem
Prompting is widely dismissed as alchemy or hacking rather than science, despite having driven many major LLM discoveries (ICL, chain-of-thought, Constitutional AI); the paper argues this is a category error.
Approach
Distinguishes "prompt science" (falsifiable, structured use of natural inputs/outputs to discover and confirm regularities) from "prompt engineering" (brute-force optimization without explanation), and argues prompting complements mechanistic interpretability via Marr's levels of analysis, language abstractions, and intervention at the native interface.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Append “let's think step-by-step” to the prompt | LLMs · math benchmarks | math score | Improved |
| Instruction-tune the model | LLMs · prompt perturbations | sensitivity to perturbations | Improved |
| Instruction-tune the model | LLMs · prompting | flexibility | Hurt |
| Use a single long prompt rather than multi-turn prompting | LLMs · prompting | effectiveness | Improved |
Surprising findings
- Prompt brittleness is argued to be a feature for science rather than only a flaw, since sensitivity reveals what influences model behavior.
Limitations
- No new experiments; claims are supported by argument and case studies from prior work.
- Acknowledges prompting's precision limits compared with mechanistic approaches.
Experimental setup
Builds on: chain-of-thought prompting in-context learning Constitutional AI Marr's levels of analysis
Closest papers by meaning
- Ask, and it shall be given: On the Turing completeness of prompting (ICLR 2025 · similarity 0.87)
- Teaching LLMs How to Learn with Contextual Fine-Tuning (ICLR 2025 · similarity 0.86)
- CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.86)
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.86)
- Do LLMs ``know'' internally when they follow instructions? (ICLR 2025 · similarity 0.86)
- Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
