Research map / Label-efficient and robust learning / Knowledge distillation
Multimodal Dataset Distillation via Phased Teacher Models
Shengbin Guo, Hang Zhao, Senqiao Yang, Chenyang Jiang, Yuhang Cheng, Xiangru Peng, Rui Shao, Zhuotao Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Shortcut Trajectory to the base (ablation) | base MTT-style distillation · Flickr30k (500 pairs) | IR@1: 12.2 -> 14.2 | Improved |
| add Phased Teacher Modeling to the base (ablation) | base MTT-style distillation · Flickr30k (500 pairs) | IR@1: 12.2 -> 13.4 | Improved |
| add EMA to the base (ablation) | base MTT-style distillation · Flickr30k (500 pairs) | IR@1: 12.2 -> 12.9 | Improved |
| add PTM to ST (ablation) | ST · Flickr30k (500 pairs) | IR@1: 14.2 -> 15.4 | Improved |
| add EMA on top of PTM + ST (ablation) | PTM + ST · Flickr30k (500 pairs) | IR@1: 15.4 -> 15.5 | Improved |
| add PTM to ST (ablation) | ST · COCO (500 pairs) | IR@1: 4.9 -> 6.3 | Improved |
| add EMA to PTM + ST (ablation) | PTM + ST · COCO (500 pairs) | TR@1: 6.2 -> 6.9 | No clear effect |
| add EMA to the base (ablation) | base MTT-style distillation · COCO (500 pairs) | TR@1: 4.2 -> 4.0 | No clear effect |
| replace LoRS with PTM-ST | LoRS · Flickr30k (500 pairs) | IR@1: 12.7 -> 16.0 | Improved |
| replace LoRS with PTM-ST | LoRS · COCO (500 pairs) | IR@1: 3.3 -> 6.6 | Improved |
| increase distilled pairs from 500 to 1000 for LoRS | LoRS · Flickr30k | IR@1: 12.7 -> 9.8 | Hurt |
| increase distilled pairs from 500 to 1000 for PTM-ST | PTM-ST · Flickr30k | IR@1: 16.0 -> 17.5 | Improved |
Closest papers by meaning
- CIFD: Controlled Information Flow to Enhance Knowledge Distillation (NeurIPS 2024 · similarity 0.87)
- Efficient Multimodal Dataset Distillation via Generative Models (NeurIPS 2025 · similarity 0.87)
- TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models (ICLR 2025 · similarity 0.86)
- See Further When Clear: Curriculum Consistency Model (CVPR 2025 · similarity 0.86)
- Scaling Multi-Teacher Distillation for Digital Pathology (NeurIPS 2026 · similarity 0.86)
- Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
