Research map / Architectures and efficiency / Neural architecture search
Minimizing the Accumulated Trajectory Error To Improve Dataset Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| regularize the real-data teacher trajectory to be flat (FTD) | MTT dataset distillation · CIFAR-10, 50 ipc | accuracy: 71.6 -> 73.8 | Helped |
| regularize the real-data teacher trajectory to be flat (FTD) | MTT dataset distillation · CIFAR-100, 10 ipc | accuracy: 39.7 -> 43.4 | Helped |
| regularize the real-data teacher trajectory to be flat (FTD) | MTT dataset distillation · ImageNet subsets, 1 and 10 ipc | accuracy: +4.5% or more on ImageNette | Helped |
| add random weight perturbations during distillation (ablation) | MTT · CIFAR-10 and CIFAR-100, 50 ipc | accuracy: +1.1% on CIFAR-10; +0.2% on CIFAR-100 | Helped |
| add random weight perturbations during distillation (ablation) | MTT · CIFAR-10 and CIFAR-100, 1 and 10 ipc | accuracy | Hurt |
| remove EMA (ablation) | FTD · CIFAR-100, 10 ipc | accuracy: 43.2 -> 43.4 | No clear effect |
| remove EMA (ablation) | FTD · CIFAR-100, 50 ipc | accuracy: 50.7 -> 49.8 | Hurt |
| remove EMA (ablation) | FTD · Tiny ImageNet, 1 and 10 ipc | accuracy: 10.0 -> 9.8; 24.5 -> 24.1 | Hurt |
| apply SAM to the distilled MTT dataset to obtain a flat minimum, rather than flattening the teacher trajectory (ablation) | MTT synthetic dataset · CIFAR-100, 10 ipc | accuracy | No clear effect |
| use FTD rather than MTT to construct the NAS proxy dataset | CIFAR-10 architecture search · top-5 Spearman rank correlation | Spearman rank correlation: 0.41 -> 0.87 | Helped |
| use FTD rather than MTT to construct the NAS proxy dataset | CIFAR-10 architecture search · top-10 and top-20 rankings | Spearman rank correlation: 0.36 -> 0.68; -0.04 -> 0.54 | Helped |
| use FTD distilled images instead of MTT images | CIFAR-10 cross-architecture evaluation · ResNet18, VGG11, and AlexNet | accuracy: 61.9 -> 65.7; 55.4 -> 58.4; 48.2 -> 53.8 | Helped |
Closest papers by meaning
- Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching (ICLR 2024 · similarity 0.92)
- Towards Stable and Storage-efficient Dataset Distillation: Matching Convexified Trajectory (CVPR 2025 · similarity 0.90)
- Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality (ICLR 2024 · similarity 0.89)
- Efficient Dataset Distillation for Pre-Trained Self-Supervised Models via Statistical Flow Matching (NeurIPS 2026 · similarity 0.89)
- Large Scale Dataset Distillation with Domain Shift (ICML 2024 · similarity 0.89)
- Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.