Research map / Architectures and efficiency / RNNs and normalization
Dataset Condensation with Gradient Matching
Bo Zhao, Konda Reddy Mopuri, Hakan Bilen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace selected real examples with gradient-matched synthetic images | ConvNet classifiers trained on condensed sets · MNIST, FashionMNIST, SVHN, CIFAR10 | accuracy: 91.7$\pm$0.5 vs 89.2$\pm$1.6; 70.5$\pm$0.6 vs 67.0$\pm$1.9; 31.2$\pm$1.4 vs 21.0$\pm$1.5; 28.3$\pm$0.5 vs 21.5$\pm$1.2 | Improved |
| increase the number of condensed images per class | Dataset Condensation · MNIST, FashionMNIST, SVHN, CIFAR10 | accuracy: 91.7$\pm$0.5 -> 98.8$\pm$0.2; 70.5$\pm$0.6 -> 83.6$\pm$0.4; 31.2$\pm$1.4 -> 82.3$\pm$0.3; 28.3$\pm$0.5 -> 53.9$\pm$0.5 | Improved |
| use a class-wise gradient-matching distance grouped by layer and output node (ablation) | Dataset Condensation · MNIST, 1 image/class, LeNet and ResNet | accuracy: 65.0$\pm$5.1 -> 85.0$\pm$1.7; 68.8$\pm$7.8 -> 89.4$\pm$0.9 | Improved |
| use InstanceNorm in the classifier trained on condensed images (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy: 79.0$\pm$2.2 -> 90.7$\pm$0.7 | Improved |
| use BatchNorm in the classifier trained on condensed images (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy: 79.0$\pm$2.2 -> 80.8$\pm$2.0 | Hurt |
| use Sigmoid instead of ReLU in the classifier (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy: around $5\%$ lower | Hurt |
| use average pooling rather than max pooling (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy: 91.7$\pm$0.5 | Improved |
| use pooling rather than no pooling (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy | Improved |
| train the synthetic images with deeper ConvNets (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy | Improved |
| increase generator network width (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy | No clear effect |
| train the classifier with 128 filters per block (ablation) | Dataset Condensation · MNIST, 1 image/class | accuracy | Improved |
| replace gradient-matched synthetic data with Herding-selected examples | Continual learning rehearsal memory · SVHN -> MNIST -> USPS, T3 | accuracy: 2.3\% | Hurt |
Closest papers by meaning
- Elucidating the Design Space of Dataset Condensation (NeurIPS 2024 · similarity 0.91)
- M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy (AAAI 2024 · similarity 0.89)
- Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustment (NeurIPS 2024 · similarity 0.88)
- Real-Fake: Effective Training Data Synthesis Through Distribution Matching (ICLR 2024 · similarity 0.87)
- Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.86)
- Fetch and Forge: Efficient Dataset Condensation for Object Detection (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
