AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation

Bowei He, Yankai Chen, Xiaokun Zhang, Linghe Kong, Philip S. Yu, Xue Liu, Chen Ma

ICLR 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Identifier (ablation)IOA (Qwen2.5-3B) · DollyEval (o1 teacher)ROUGE-L: 36.72 -> 38.16Improved
add Organizer curriculum (ablation)IOA (Qwen2.5-3B) · GSM8K (o1 teacher)Pass@1: 51.68 -> 55.79Improved
add Adapter (ablation)IOA (Qwen2.5-3B) · HumanEval (o1 teacher)Pass@1: 36.48 -> 40.64Improved
add Adapter (ablation)IOA (LLaMA3.2-3B) · HumanEval (o1 teacher)Pass@1: 34.42 -> 37.98Improved
add Organizer curriculum (ablation)IOA (LLaMA3.2-3B) · GSM8K (o1 teacher)Pass@1: 48.92 -> 52.07Improved
add Identifier (ablation)IOA (LLaMA3.2-3B) · DollyEval (o1 teacher)ROUGE-L: 35.47 -> 36.88Improved
add Identifier (ablation)IOA (Qwen2.5-3B) · HumanEval (o1 teacher)Pass@1: 39.87 -> 40.64Improved
use pedagogical IOA instead of best baseline MADAQwen2.5-3B · HumanEval (o1 teacher)Pass@1: 33.39 -> 40.64Improved
switch teacher from o1 to DeepSeek-R1IOA (Qwen2.5-3B) · MATHPass@1: 15.53 -> 16.02Improved
use higher-quality 3K seed dataIOA (Qwen2.5-3B) · AIME2024Pass@1: 6.29 -> 27.52Improved
increase seed data amount to 9K synthesized samplesIOA (Qwen2.5-3B) · AIME2024Pass@1: 6.29 -> 11.83Improved
increase tau_ZPD beyond 0.15IOA · averaged ROUGE-L and Pass@1averageMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy