Research map / Large language models / Agents, code and math
FEEDBACK FRICTION: LLMs Struggle to Fully Incorporate External Feedback
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a controlled iterative feedback setup showing LLM solvers plateau below their target accuracy even with near-perfect ground-truth-informed feedback ("Feedback Friction"), with partial mitigation from temperature schedules and answer rejection sampling.
Problem
Whether LLMs can fully incorporate near-perfect external feedback during test-time iterative self-improvement, and what limits their ability to do so.
Approach
Decouples feedback quality from feedback incorporation by using a feedback generator with access to ground-truth answers and solution steps (answers masked), then measures how far solver accuracy falls below a computed target ceiling. It names the resulting plateau "Feedback Friction" and links it to semantic-entropy confidence and to a gap between stated and actual belief updating.
Key results
- Across nine tasks, solver models such as Claude 3.7 Thinking plateau below their theoretical target accuracy even with strong (GPT-4.1 mini) feedback, e.g. 15-25% below ceiling on AIME after 10 iterations.
- Feedback resistance accounts for 62.8-100% of unsolved errors after 10 iterations, per o4-mini classification validated at 96% agreement with humans.
- Models claim to understand feedback (>95% yes) and willingness to update (>96% yes) yet fail to incorporate it; absolute improvement rate rises with semantic entropy from near-zero at high confidence to 0.4-0.8 at high entropy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace binary correctness feedback with strong-model reflective feedback | Llama-4-Maverick · AIME 2024 | accuracy: +26.7% | Helped |
| Replace binary correctness feedback with strong-model reflective feedback | Llama-4-Maverick · GPQA | accuracy: +10.6% | Helped |
| Replace binary correctness feedback with strong-model reflective feedback | Llama-4-Scout · AIME 2024 | accuracy: +33.3% | Helped |
| Replace binary correctness feedback with strong-model reflective feedback | Llama-4-Scout · GPQA | accuracy: +13.1% | Helped |
| Replace binary correctness feedback with strong-model reflective feedback | Llama-3.3-70B · AIME 2024 | accuracy: +26.7% | Helped |
| Replace binary correctness feedback with strong-model reflective feedback | Llama-3.3-70B · GPQA | accuracy: +6.6% | Helped |
| Increase sampling temperature progressively across feedback iterations (ablation) | Llama-4-Scout and Llama-4-Maverick · multiple tasks | accuracy | Helped |
| Combine progressive temperature increases with rejection of previously attempted answers (ablation) | Llama-4-Scout and Llama-4-Maverick · multiple-choice and non-multiple-choice tasks | accuracy | Helped |
| Use multi-turn conversation formatting instead of a single prompt (ablation) | Solver models in the iterative feedback pipeline · multiple tasks | accuracy | No clear effect |
| Provide reflective feedback rather than binary correctness feedback | Llama model families · AIME 2024, MMLU Pro, and GPQA | accuracy | Helped |
| Increase feedback opportunities through iterative correction with strong-model reflective feedback | Frontier solver models · all evaluated tasks | accuracy relative to theoretical target | Hurt |
| Use higher semantic entropy (lower-confidence answers) as the question subset | Llama-3.3-70B and Llama-4-Scout · 5-digit multiplication, GPQA, MATH, MMLU Pro, and TriviaQA | absolute improvement rate: 0.4-0.8 | Helped |
Surprising findings
- Models say they understand feedback and will update their beliefs (>95%) yet do not incorporate it in the next attempt.
- Highly confident (low semantic entropy) predictions show near-zero improvement from identical-quality feedback.
Limitations
- No mechanistic explanation for feedback resistance is provided.
- Mitigation via sampling yields only modest gains; fine-tuning was not attempted due to compute constraints.
- Subsampled 10% of MMLU, MMLU Pro, PopQA, and TriviaQA for cost reasons; Claude experiments limited to fewer analyses.
Experimental setup
Datasets: AIME 2024 MATH-500 TriviaQA PopQA MMLU MMLU Pro GPQA 5-digit multiplication (synthetic) hexadecimal 5-digit multiplication (synthetic)
Benchmarks: AIME 2024 MATH-500 TriviaQA PopQA MMLU MMLU Pro GPQA 5-digit multiplication hexadecimal 5-digit multiplication
Metrics: accuracy absolute improvement rate semantic entropy buckets
Baselines: Binary Correctness Feedback Self-Generated Reflective Feedback temperature-only sampling
Builds on: semantic entropy lm-evaluation-harness
Closest papers by meaning
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing (NeurIPS 2024 · similarity 0.89)
- Learning to Route LLMs with Confidence Tokens (ICML 2025 · similarity 0.88)
- GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements (ICML 2024 · similarity 0.88)
- MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback (ICLR 2024 · similarity 0.88)
- Rethinking Thinking Tokens: LLMs as Improvement Operators (ICML 2026 · similarity 0.88)
- Advancing LLM Reasoning with Natural Language and Numerical Feedback (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.