Research map / Label-efficient and robust learning / Knowledge distillation
Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Montessori-Instruct trains a teacher LLM with DPO on preference pairs derived from local data influence on a student model, so the teacher generates more student-beneficial synthetic instructions.
Problem
Synthetic instruction data from a teacher LLM is often noisy, non-informative, or misleading for the student, and standard synthesis ignores what the student actually learns from.
Approach
Measure each synthetic data point's local data influence on the student (change in reference loss after one optimizer step on that point), build preference pairs of instructions with positive vs. negative influence, and optimize the teacher's instruction-generation ability with DPO; iterate to track the student's evolving preferences.
Key results
- On Llama3-8B student, Montessori-Instruct (iteration 2) reaches 56.82% LC-WR and 60.23% WR on AlpacaEval 2.0 vs. 50% for Self-Instruct baseline.
- MT-Bench score of 7.092 for Montessori-Instruct iteration 2 vs. 6.490 for Self-Instruct on the 8B setting.
- Montessori-Instruct outperforms Self-Instruct with GPT-4o teacher (54.95% LC-WR) on AlpacaEval 2.0 in the 8B setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize the teacher using student-preference-guided local influence | Llama3-8B student training data · Alpaca Eval 2.0 | WR: 50% -> 60.23% | Helped |
| Optimize the teacher using student-preference-guided local influence | Llama3-8B student training data · MT-Bench | score: 6.490 -> 7.092 | Helped |
| Optimize the teacher using student-preference-guided local influence | TinyLlama-1.1B student training data · MT-Bench | score: 2.154 -> 2.504 | Helped |
| Replace local data influence with LLM-as-a-Judge scoring (ablation) | Montessori-Instruct teacher optimization · Alpaca Eval 2.0 | WR: 58.59% -> 54.93% | Hurt |
| Replace local data influence with training loss (ablation) | Montessori-Instruct teacher optimization · MT-Bench | score: 6.903 -> 6.656 | Hurt |
| Bootstrap top 50% influential data instead of optimizing teacher parameters (ablation) | Montessori-Instruct · Alpaca Eval 2.0 | WR: 58.59% -> 48.14% | Hurt |
| Optimize responses rather than instructions (ablation) | Montessori-Instruct teacher · MT-Bench | score: 6.903 -> 6.556 | Hurt |
| Use Open Assistant out-of-domain seed data (ablation) | Montessori-Instruct · MT-Bench | score: 6.903 -> 6.706 | Hurt |
| Use Alpaca Eval test examples as seed data (ablation) | Montessori-Instruct · Alpaca Eval 2.0 | WR: 58.59% -> 61.36% | Helped |
Surprising findings
- Data from a weaker teacher (Llama3-8B-Instruct) tailored to the student outperforms data from the stronger GPT-4o teacher with standard Self-Instruct.
- Original Self-Reward without an external judge fails on Llama3-8B and TinyLlama-1.1B, producing near-zero win rates, because the small students cannot judge their own responses.
Limitations
- Evaluated only at 10K synthetic examples; effectiveness at production scale (~100K) is unclear.
- Substantial computational overhead from per-data-point influence collection (one-step training and evaluation per sample).
- Evaluation relies on GPT-4-based judging for AlpacaEval and MT-Bench.
Experimental setup
Datasets: Alpaca GPT-4 (52K seed pool) Alpaca Eval Open Assistant
Benchmarks: AlpacaEval 2.0 MT-Bench MMLU GPQA ARC-Challenge GSM8K HellaSwag
Metrics: length-controlled win rate win rate MT-Bench score accuracy
Baselines: Self-Instruct Self-Instruct with GPT-4o teacher Self-Reward LLM2LLM
Builds on: Self-Instruct Direct Preference Optimization influence functions LLM2LLM Self-Reward
Closest papers by meaning
- DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs (ICML 2025 · similarity 0.86)
- Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming (NeurIPS 2024 · similarity 0.85)
- Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback (NeurIPS 2024 · similarity 0.85)
- SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning (NeurIPS 2024 · similarity 0.85)
- MiniPLM: Knowledge Distillation for Pre-training Language Models (ICLR 2025 · similarity 0.84)
- MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.