AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Failure Profiling and Reachable Trajectory Selection for Reasoning Distillation

yijun Zhu, Jianxin Wang, Chengchao Shen

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes failure-profiling and reachable-trajectory selection to improve reasoning distillation from a larger teacher to a smaller student for math reasoning.

Problem

Students distilled from a larger teacher's reasoning solutions overfit the teacher's output distribution and generalize poorly when their own inference-time behavior deviates from the training data; prior error-data methods only partially address this.

Approach

Profiles the student's failure modes and selects teacher trajectories that the student can actually reach (reachable trajectories), rather than simply adding error data or using all teacher outputs. Details beyond the abstract are not available.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add behavioral failure profiling with mode-specific teacher prompts (ablation)FPRT · Five-benchmark average, Llama-3.1-8Baccuracy (%): 72.9 -> 79.0Improved
add branching point reachability (BPR) selection instead of global log-prob scoring (ablation)FPRT · Five-benchmark average, Llama-3.1-8Baccuracy (%): 77.3 -> 79.0Improved
add BPR on top of uniform prompting (no profiling) (ablation)FPRT · Five-benchmark average, Llama-3.1-8Baccuracy (%): 72.4 -> 72.9No clear effect
BPR scoring versus global scoring (ablation)FPRT · MAWPS, Llama-3.1-8Baccuracy (%): 93.9 vs 94.2 globalHurt
select by BPR instead of random correct candidate (ablation)FPRT · Five-benchmark average, Llama-3.1-8Baccuracy (%): 76.6 -> 79.0Improved
select by BPR instead of random correct candidate (ablation)FPRT · Five-benchmark average, DeepSeek-Math-7Baccuracy (%): 79.0 -> 80.0Improved
use full failure-profiled pipeline instead of direct teacher-trace SFT (ablation)FPRT · Five-benchmark average, Llama-3.1-8Baccuracy (%): 75.9 -> 79.0Improved
set BPR window to 8 sentences instead of shorter or longer (ablation)FPRT · Five-benchmark average, Llama-3.1-8Baccuracy (%): L=2 77.1, L=4 76.7, L=8 79.0, L=16 78.6, L=32 78.2Improved
scale corpus from 16.5K to 88.90K examplesFPRT · Llama-3.1-8B averageaccuracy (%): 71.8 -> 79.7Improved
scale corpus from 16.5K to 88.90K examplesFPRT · DeepSeek-Math-7B averageaccuracy (%): 77.9 -> 80.6Improved
FPRT versus LEMMA on competition-level mathFPRT · MATH, Llama-3.1-8Baccuracy (%): 44.3 vs 48.8 (62.74K); 46.2 at 88.90KHurt
switch teacher from DeepSeek-V3.2 to DeepSeek-V4-Flash (ablation)FPRT · DeepSeek-Math-7B averageaccuracy (%): 80.6 -> 81.0No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy