Research map / Large language models / Reasoning and chain of thought

Collaborative Reasoning Distillation via Cross-Feedback and Coherent Curation

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Collaborative Reasoning Distillation (CRD), which combines cross-feedback between two teachers, dual-judge step scoring, and Viterbi-style step stitching to build a 50K curated dataset, then trains students with RQO.

Problem

Distilling reasoning into small LLMs typically relies on outcome-based correctness signals, which cannot distinguish sound reasoning from lucky guesses, and single-teacher traces inherit teacher biases and limited diversity.

Approach

Teachers critique and refine each other's chains over rounds, each step is scored by two LLM judges (JudgeLM and xVerify) with trimmed-mean aggregation, and a Viterbi-style DP stitches high-scoring, coherent steps across teachers into hybrid chains; students are trained with a quality-margin-weighted preference loss (RQO) and a curriculum token budget.

Key results

• CRD-1.5B reaches 91.1% on MATH-500 vs 83.9% for DeepSeek-R1-Distill-Qwen-1.5B and 35.1% on AIME'25 vs ProRL-1.5B's 33.3%.
• CRD-30B-A3B scores 82.5% on AIME'24 and 73.2% on AIME'25, above its Qwen3-30B-A3B base (80.4%, 70.9%).
• In the same-data comparison on Qwen2.5-1.5B, CRD reaches 91.1% on MATH-500 vs MAGDi's 72.6%.

Surprising findings

• CRD-30B-A3B is reported to surpass its teacher DeepSeek-R1 on math and code benchmarks despite far fewer active parameters.
• Naive SFT on the same 50K trajectories already reaches 90.5% on MATH-500, so most of the gain is attributed to data curation rather than RQO.

Limitations

• Relies on LLM teachers and judges that may introduce biases, so plausible but incorrect steps can receive high scores.
• Only one teacher pair (DeepSeek-R1 and Gemini 2.5 Flash) is evaluated, and RQO is not compared against DPO.
• Training cost reported covers student optimization only, excluding teacher generation and curation.

Experimental setup

Datasets: DeepScaleR Eurus-2-RL Reasoning Gym SCP-116K

Benchmarks: MATH-500 AIME'24 AIME'25 LiveCodeBench GPQA-Diamond GSM8K HumanEval BoxNet HLE

Metrics: pass@1 pass@2 pass@4 accuracy

Baselines: DeepSeek-R1-Distill-Qwen-1.5B ProRL-1.5B Gemini-Flash-Distill-1.5B TinyLLM FAIR MAGDi Naive SFT Qwen3-30B-A3B

Builds on: DeepSeek-R1 GRPO DPO xVerify JudgeLM

Closest papers by meaning

• CoRe: Collaborative Reasoning via Cross Teaching (ICML 2026 · similarity 0.91)
• RLKD: Distilling LLMs’ Reasoning via Reinforcement Learning (AAAI 2026 · similarity 0.89)
• The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation (ICLR 2026 · similarity 0.89)
• The Signal is in the Steps: Local Scoring for Reasoning Data Selection (ICML 2026 · similarity 0.89)
• Reasoning Scaffolding: Distilling the Flow of Thought from LLMs (ICLR 2026 · similarity 0.89)
• Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation (NeurIPS 2026 · similarity 0.88)