AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

From Solver Trajectories to Teaching Trajectories: Cognition-Aligned Reasoning Distillation

Yashuo Luo, Qianren Mao, ZiqiQin, Junnan Liu, Haitao Li, Qili Zhang, Yuan Gao, Zhijun Chen, Xu Wang

NeurIPS 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Argues that raw solver trajectories from large reasoning models are a poor distillation target and proposes cognition-aligned teaching trajectories for reasoning distillation.

Problem

Knowledge distillation from large reasoning models (LRMs) typically uses raw teacher trajectories as supervision, which mismatches the goal of distillation because LRMs are optimized as solvers via reward-driven post-training rather than as teachers.

Approach

Reframes distillation supervision from solver trajectories to teaching trajectories that are aligned with student cognition, rather than reusing the teacher's raw reward-optimized reasoning traces.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
reformat failed-step trajectories with CARD instead of raw SFTQwen2.5-7B student · Math averageaccuracy: 51.80 -> 54.22Improved
reformat failed-step trajectories with CARD instead of raw SFTQwen2.5-7B student · General OOD averageaccuracy: 56.34 -> 71.07Improved
reformat with CARD instead of raw SFTLlama3.2-1B student · Math averageaccuracy: 11.41 -> 13.77Improved
add Logical Coherence Reward (ablation)CARD Reformatter · General OOD average (Qwen2.5-7B)accuracy: 66.31 -> 71.07Improved
add answer-distribution KL reward (ablation)CARD Reformatter · Math average (Qwen2.5-7B)accuracy: 52.07 -> 54.22Improved
add rationale-distribution KL reward (ablation)CARD Reformatter · Math average (Qwen2.5-7B)accuracy: 52.23 -> 54.22Improved
add leakage penalty (ablation)CARD Reformatter · Math average (Qwen2.5-7B)accuracy: 52.66 -> 54.22Improved
reformat only FSF>0 trajectories instead of random selection (ablation)CARD · General OOD averageaccuracy: 52.66 -> 71.07 (Qwen2.5-7B)Improved
reformat failed-step trajectories instead of dropping them (ablation)Vanilla SFT · Math average (Qwen2.5-7B)accuracy: 51.80 -> 50.93Hurt
replace failed-step trajectories with clean ones (Replace-FSF) (ablation)Vanilla SFT · Math average (Qwen2.5-7B)accuracy: 51.80 -> 52.74Improved
replace failed-step rationales with TokenSkip compressionVanilla SFT · Math average (Llama3.2-3B)accuracy: 31.45 -> 29.36Hurt
flip 20% of FSF judge decisions (ablation)CARD · General OOD average (Qwen2.5-7B)accuracy: 71.07 -> 66.42Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy