Research map / Large language models / AI and society
Understanding the Effects of Iterative Prompting on Truthfulness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies how iterative re-prompting affects GPT-3.5's truthfulness on TruthfulQA, finds naive "Are you sure?" prompting hurts accuracy and calibration via apologetic flips, and proposes two improved iteration prompts that raise accuracy and beat Self-Consistency baselines.
Problem
Iterative prompting is assumed to refine LLM answers, but its effect on truthfulness and calibration is unclear; naive re-prompting may reduce accuracy.
Approach
Analyzes naive "Are you sure?" iterative prompting, attributes accuracy decline to apologetic answer flips and rising overconfidence, and designs two iteration prompts (re-asking the question; asking for supporting facts then refining) that avoid apologies and improve accuracy over iterations.
Key results
- Naive "Are you sure?" iterative prompting drops GPT-3.5 TruthfulQA accuracy from about 70% to about 45% after one iteration, with incorrect flips (~32.5%) far exceeding correct flips (~12.5%).
- Improved Prompt-1 raises accuracy from 69.1% to 74.2% over iterations, and Improved Prompt-2 from 68.7% to 73.7%.
- Improved Prompt-1 reaches ~73% accuracy versus ~66% for Self-Consistency and ~62.6% for Universal Self-Consistency.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Re-prompt with “Are you sure?” | GPT-3.5 Turbo · TruthfulQA MC1 | accuracy: approximately 70% to around 45% | Hurt |
| Re-prompt with “Are you sure?” | GPT-3.5 Turbo · TruthfulQA MC1 | ECE: 0.17 to 0.30 | Hurt |
| Repeat the question as the iteration prompt | GPT-3.5 Turbo iterative prompting · TruthfulQA MC1 | accuracy: 69.1% to 74.2% | Helped |
| Ask for supporting facts and refine the answer | GPT-3.5 Turbo iterative prompting · TruthfulQA MC1 | accuracy: 68.7% to 73.7% | Helped |
| Use Improved Prompt-1 instead of Self-Consistency | GPT-3.5 Turbo · TruthfulQA MC1 | accuracy: ∼73% compared to ∼66% | Helped |
| Use Improved Prompt-1 instead of Universal Self-Consistency | GPT-3.5 Turbo · TruthfulQA MC1 | accuracy: ∼73% compared to ∼62.6% | Helped |
| Replace “Are you sure?” with the self-corrective iteration prompt (ablation) | Naive iterative prompting · TruthfulQA MC1 | accuracy | No clear effect |
| Replace “Are you sure?” with explicit self-improvement instructions (ablation) | Naive iterative prompting · TruthfulQA MC1 | accuracy | No clear effect |
| Add an explanation request and “After further investigation” instruction (ablation) | Improved Prompt-2 · TruthfulQA MC1 | accuracy | No clear effect |
| Explicitly request new facts, list them, then answer (ablation) | Improved Prompt-2 · TruthfulQA MC1 | accuracy | No clear effect |
Surprising findings
- Naive re-prompting makes the model apologize in 100% of responses and flip correct answers to incorrect ones, with ECE spiking from 0.17 to 0.30 after re-evaluation.
- Simply re-asking the question as the iteration prompt, rather than asking whether it is sure, avoids the accuracy decline and improves accuracy with iterations.
Limitations
- Experiments limited to a single model (GPT-3.5) and a limited set of prompts, so patterns may not generalize.
- Evaluation restricted to the multiple-choice mc1 subset of TruthfulQA.
- Causal link between apologies and accuracy decline is inferred from observed patterns rather than directly tested.
Experimental setup
Datasets: TruthfulQA
Benchmarks: TruthfulQA
Metrics: accuracy expected calibration error answer flips apology rate
Baselines: Naive iterative prompting ("Are you sure?") Self-Consistency Universal Self-Consistency Random selection Greedy selection
Builds on: Self-Consistency Universal Self-Consistency TruthfulQA Expected Calibration Error
Closest papers by meaning
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.89)
- HonestLLM: Toward an Honest and Helpful Large Language Model (NeurIPS 2024 · similarity 0.88)
- Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing (ICLR 2024 · similarity 0.87)
- Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.87)
- Evaluating Large Language Models at Evaluating Instruction Following (ICLR 2024 · similarity 0.87)
- Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales? (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.