Research map / Large language models / Reasoning and chain of thought

Automatic Curriculum Expert Iteration for Reliable LLM Reasoning

ICLR 2025 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Auto-CEI, which combines expert iteration with an automatically updated reward curriculum (via hill-climbing on c1) so an LLM answers assertively within its reasoning limits and says "I don't know" only after sufficient reasoning steps.

Problem

LLMs hallucinate on multi-step reasoning beyond their capability and, when over-corrected, become lazy by refusing answerable questions; a method is needed to balance assertiveness and refusal in reasoning.

Approach

Uses the number of reasoning steps as a proxy for difficulty and designs a reward curriculum that gives refusals a compensation only after long reasoning, tuned by an objective f = (1-λ)Precision + λ(1-IDK rate) via hill climbing, with expert iteration resampling trajectories under that reward.

Key results

• On BoardgameQA, Auto-CEI reaches 84.52% precision with 29.37% IDK and f=0.817 (λ=0.2), versus 80.77% precision for EI+R-Tuning.
• On MATH, Auto-CEI achieves f=0.575 with 55.63% precision and 36.08% IDK, but its accuracy (35.56%) is below Vanilla EI (42.04%).
• On Blocksworld, Auto-CEI reaches 91.53% precision with 18.30% IDK and 74.78% accuracy, outperforming SFT+R-Tuning and RLKF in f.

Measured effects reported in the paper

Surprising findings

• RLKF (DPO) degraded reasoning performance, with accuracy dropping below SFT on BoardgameQA and Blocksworld.
• On MATH, Auto-CEI has lower accuracy than Vanilla EI (35.56% vs 42.04%), so its gains are in precision/refusal balance rather than raw accuracy.

Limitations

• Complexity is proxied by number of reasoning steps, which may not hold for simpler problems or problems with short optimal solutions.
• Model sometimes refuses correct answers; method requires multiple expert iteration rounds and is costly to optimise.
• Evaluated on only three benchmarks, with additional generalization tested on a single model (Mistral) for BoardgameQA only.

Experimental setup

Datasets: BoardgameQA MATH Blocksworld

Benchmarks: BoardgameQA MATH Blocksworld

Metrics: accuracy precision refusal rate (IDK) objective f refusal accuracy

Baselines: SFT Vanilla EI SFT + R-Tuning EI + R-Tuning RLKF (DPO)

Builds on: Expert Iteration R-Tuning RLKF Curriculum reinforcement learning

Closest papers by meaning

• Toward Adaptive Reasoning in Large Language Models with Thought Rollback (ICML 2024 · similarity 0.88)
• Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Following (AAAI 2026 · similarity 0.88)
• DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search (ICLR 2025 · similarity 0.87)
• Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory? (ICLR 2026 · similarity 0.87)
• Mitigating Hallucinations in Large Language Models via Causal Reasoning (AAAI 2026 · similarity 0.87)
• Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models (NeurIPS 2025 · similarity 0.87)