Research map / Image generation / Autoregressive and latent generation
Color-Oriented Redundancy Reduction in Dataset Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoPalette, which distills datasets into 4-bit/64-color images via a learned palette network and selects initialization images with graph-cut information gain on quantized images to reduce color redundancy.
Problem
Dataset distillation stores synthetic images at full 8-bit color, leaving considerable color-space redundancy within images and across images of a class, which wastes the storage budget.
Approach
A palette network assigns each pixel to one of K reduced-color buckets, trained with the distillation loss plus maximum-color and palette-balance losses; initialization picks diverse real images via graph-cut information gain computed on median-cut quantized images.
Key results
- On CIFAR100 with ConvNetD3, AutoPalette reaches 52.6% at IPC=10 and 53.3% at IPC=50, reported as 6.7% and 4.2% above prior parameterization SOTA.
- On CIFAR10 with ConvNetD3 at IPC=10, AutoPalette achieves 74.3% versus 73.5% for SPEED.
- Applied to trajectory matching, the palette framework raises CIFAR100 IPC=1 accuracy from 24.3% to 38.0%; applied to distribution matching it adds up to 15% on CIFAR100 IPC=10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add maximum color loss (ablation) | AutoPalette palette network · CIFAR10, IPC 10 | accuracy (%): 64.00 -> 66.20 | Helped |
| add palette balance loss (ablation) | AutoPalette palette network · CIFAR10, IPC 10 | accuracy (%): 61.40 -> 66.20 | Helped |
| add alignment loss (ablation) | AutoPalette palette network · CIFAR10, IPC 10 | accuracy (%): 60.14 -> 66.20 | Helped |
| use graph-cut selection on full-color images instead of random real-image initialization (ablation) | AutoPalette initialization · CIFAR10, IPC 10 | accuracy (%): 60.84 -> 61.41 | Helped |
| use graph-cut selection on quantized images instead of full-color images (ablation) | AutoPalette initialization · CIFAR10, IPC 10 | accuracy (%): 61.41 -> 62.13 | Helped |
| reduce the palette from 256 to 64 colors (ablation) | AutoPalette · CIFAR10, IPC 10 | accuracy (%): 66.8 -> 66.2 | No clear effect |
| reduce colors from 64 to 2 per image (ablation) | AutoPalette · CIFAR10, IPC 10 | accuracy (%): 66.2 -> 55.3 | Hurt |
| use the palette network instead of Median Cut quantization (ablation) | CIFAR10 synthetic dataset, IPC 10 · CIFAR10, IPC 10; scale 64 | accuracy (%): 60.7 -> 66.2 | Helped |
| use the palette network instead of OCTree quantization (ablation) | CIFAR10 synthetic dataset, IPC 10 · CIFAR10, IPC 10; scale 64 | accuracy (%): 60.8 -> 66.2 | Helped |
| add AutoPalette to vanilla distribution matching | DM on CIFAR10, IPC 10 · CIFAR10, IPC 10 | accuracy (%): 48.9$\pm$0.6 -> 60.9$\pm$0.1 | Helped |
| add AutoPalette to vanilla trajectory matching | TM on CIFAR100, IPC 1 · CIFAR100, IPC 1 | accuracy (%): 24.3$\pm$0.3 -> 38.0$\pm$0.1 | Helped |
Surprising findings
- Training the palette network with the task loss alone leaves most palette buckets inactive, motivating the maximum color and palette balance losses.
- Even at 2 colors per pixel, accuracy remains 55.3% on CIFAR10 IPC=10, showing strong robustness to aggressive color reduction.
Limitations
- Uses a fixed color budget across all classes, ignoring class-wise color usage bias.
- Performance drops on fine-grained ImageNet subsets where fine details are blurred by color reduction.
- Results primarily use trajectory matching; generalization to other DD frameworks is shown on fewer settings.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet subsets (ImageNette, ImageWoof, ImageFruit, ImageMeow, ImageSquawk, ImageYellow)
Benchmarks: CIFAR-10 CIFAR-100 ImageNet subsets
Metrics: test accuracy
Baselines: Random Herding K-Center DD DM DC TM DATM IDC HaBa RTP SPEED
Builds on: trajectory matching distribution matching IDC generalized graph cut submodular functions
Closest papers by meaning
- Rethinking Dataset Distillation for Classification: Do Distilled Sets Outperform Coresets? (NeurIPS 2026 · similarity 0.86)
- Large Scale Dataset Distillation with Domain Shift (ICML 2024 · similarity 0.86)
- Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustment (NeurIPS 2024 · similarity 0.85)
- DD-Ranking: Rethinking the Evaluation of Dataset Distillation (NeurIPS 2026 · similarity 0.85)
- Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios (CVPR 2025 · similarity 0.85)
- Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.