AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Critique-Guided Distillation for Robust Reasoning via Refinement

Berkcan Kapusuzoglu, Supriyo Chakraborty, Zain Sarwar, Chia-Hsuan Lee, Sambit Sahu

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning for reasoning data · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train student to refine answers conditioned on teacher critiques (CGD) instead of generating critiques (CFT)LLaMA3.1-8B Instruct · Math reasoning avg (Group 1)accuracy: 41.5 -> 46.9Improved
train with CGD instead of CFTS1.1-3B · Math reasoning avg (Group 1)accuracy: 38.9 -> 46.1Improved
use CGD instead of CFT on instruction-followingLLaMA3.1-8B Instruct · IFEvalaccuracy: 55.6 -> 76.1Improved
add critique conditioning during training (ablation)CGD · 8 math and reasoning benchmarksaccuracy: up to 114% relative drop without critiqueImproved
use specific relevant critiques instead of a generic string (ablation)CGD · AIME24 (10K subset)accuracy: 7.8 -> 12.2Improved
use irrelevant critiques instead of generic ones (ablation)CGD · AMC23 (10K subset)accuracy: 26.7 -> 18.3Hurt
mix 50/50 correct and incorrect student answers in training data (ablation)CGD · Average accuracy, 25K dataaccuracy: 36.8 -> 39.9Improved
raise learning rate from 1e-6 to 5e-6 (ablation)CFT · Average over 6 benchmarksaccuracy: 39.3 -> 30.1Hurt
raise learning rate from 1e-6 to 5e-6 (ablation)CGD · Average over 6 benchmarksaccuracy: 44.8 -> 42.2Hurt
apply CGD to LLaMA3.1-8B InstructLLaMA3.1-8B Instruct · AIME 2024 greedyPass@1: 3.3% -> 16.7%Improved
swap LLaMA3.3-70B teacher for S1.1-32B teacher (ablation)CGD on LLaMA3.1-8B Instruct · WebInstruct, math and general avgaccuracy: Math avg 46.9 -> 48.0; General avg 36.7 -> 35.7Mixed
use weaker S1.1-32B teacher instead of Claude Sonnet 3.7 (ablation)CGD on Qwen2.5-Math-7B · Math avgaccuracy: 50.4 -> 49.0No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy