Research map / Architectures and efficiency / RNNs and normalization
Multisize Dataset Condensation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Multisize Dataset Condensation (MDC), which folds N condensation processes into one by adding an adaptive subset loss that updates the most learnable subset each round, yielding multiple dataset sizes from a single condensed set.
Problem
Dataset condensation yields one fixed size, and subsets selected from a condensed set are much worse than condensing directly to the smaller size (the "subset degradation problem"); producing multiple sizes for on-device use normally requires N separate condensation runs.
Approach
Compress N condensation processes into one by adding an adaptive subset loss computed on the Most Learnable Subset (MLS), selected each outer loop by the rate of change of feature distance between subsets and the real data, with a freezing rule to protect previously learned smaller subsets.
Key results
- On CIFAR-10 at IPC 10, the method reaches 58.37% average accuracy vs 53.15% for Baseline-C (+5.22%), with up to +6.40% across runs.
- For the 1-image subset on CIFAR-10 IPC 10, accuracy improves from 27.49% to 49.66% over Baseline-C (+22.17%).
- Adds about 3.6 hours of training (11.5h to 15.1h) for IDC, but matches IDC's average accuracy in 0.6 hours, a claimed 94.8% reduction in time for the same performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adaptive subset loss and MLS selection | IDC-based multisize condensation · CIFAR-10, IPC 10 | average accuracy: 53.15 -> 58.37; +5.22 | Helped |
| Add adaptive subset loss and MLS selection | IDC-based multisize condensation · SVHN, IPC 10 | average accuracy: 70.65 -> 76.75; +6.10 | Helped |
| Add adaptive subset loss and MLS selection | IDC-based multisize condensation · CIFAR-100, IPC 10 | average accuracy: 34.40 -> 37.53; +3.13 | Helped |
| Add subset loss (ablation) | Baseline-C on CIFAR-10 · CIFAR-10, IPC 10 | average accuracy: 53.15 -> 57.08 | Helped |
| Add feature-distance comparison to select MLS (ablation) | MDC with feature-distance subset loss · CIFAR-10, IPC 10 | average accuracy: 57.08 -> 58.08 | Helped |
| Add freezing of the preceding MLS (ablation) | MDC with feature-distance comparison · CIFAR-10, IPC 10 | accuracy at IPC 1: 40.12 -> 49.66 | Helped |
| Use gradient distance rather than feature distance for subset evaluation (ablation) | MDC MLS selection · CIFAR-10, IPC 10 | average accuracy: 58.37 -> 58.73 | No clear effect |
| Use accuracy difference rather than feature distance for subset evaluation (ablation) | MDC MLS selection · CIFAR-10, IPC 10 | average accuracy: 58.37 -> 59.76 | Helped |
| Apply MDC using DREAM instead of IDC as the basic condensation method | MDC · CIFAR-10, IPC 10 | average accuracy: 58.37 -> 60.19 | Helped |
| Use class-wise MLS sizes rather than uniform MLS sizes | MDC · CIFAR-10, IPC 10 | average accuracy: 58.37 -> 59.19 | Mixed |
Surprising findings
- Adding the subset loss reduces full-size (IPC 10) accuracy slightly relative to the basic condensation in some settings, which the authors list as a limitation.
- Gradient-distance subset selection gives similar accuracy to feature distance (58.73 vs 58.37), while accuracy-difference selection is more accurate but computationally impractical.
Limitations
- The subset loss affects the accuracy of the full synthetic dataset, so the full-size result can degrade.
- The reason MDC learns faster than prior methods is not fully explained.
- Subset selection metrics are heuristic; better metrics are left open.
Experimental setup
Datasets: SVHN CIFAR-10 CIFAR-100 ImageNet-10
Benchmarks: SVHN CIFAR-10 CIFAR-100 ImageNet-10
Metrics: top-1 accuracy average accuracy over IPC subsets accuracy difference vs Baseline-C
Baselines: IDC DREAM DC DSA MTT Slimmable Dataset Condensation LBS LFS
Builds on: IDC gradient matching DREAM
Closest papers by meaning
- Elucidating the Design Space of Dataset Condensation (NeurIPS 2024 · similarity 0.87)
- Are Large-scale Soft Labels Necessary for Large-scale Dataset Distillation? (NeurIPS 2024 · similarity 0.85)
- Rethinking Dataset Distillation for Classification: Do Distilled Sets Outperform Coresets? (NeurIPS 2026 · similarity 0.84)
- Ameliorate Spurious Correlations in Dataset Condensation (ICML 2024 · similarity 0.84)
- Adaptive Dataset Quantization (AAAI 2025 · similarity 0.83)
- M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.