Research map / Architectures and efficiency / Quantization
Adaptive Dataset Quantization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adaptive Dataset Quantization (ADQ), which adaptively samples from DQ bins using texture-level representativeness and contrastive-learning diversity scores to compress datasets.
Problem
Dataset compression methods are limited: dataset distillation is computationally expensive and generalizes poorly across architectures, while coreset selection has low data keep ratios and relies on heuristics; the naive Dataset Quantization (DQ) samples uniformly from bins, ignoring their varying i
Approach
Scores each DQ bin with a texture-level representativeness score and a contrastive-learning diversity score, combines them into an importance score, and samples from each bin in proportion to importance and bin size instead of uniformly.
Key results
- On CIFAR-10 with ResNet-18 at 10%/20%/30% keep ratios, ADQ reaches 86.2/90.4/94.2% vs DQ 84.1/87.6/91.0%.
- ADQ averages +2.6%, +2.8%, +3.3% over DQ across five architectures on CIFAR-10 at 10/20/30% keep ratios.
- ADQ takes about 1 GPU hour for dataset generation, roughly 1.1% of DM's 91 GPU hours.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace uniform sampling with adaptive sampling using bin importance | DQ (ADQ) · CIFAR-10, 10% keep ratio | accuracy: +2.6 | Helped |
| replace uniform sampling with adaptive sampling using bin importance | DQ (ADQ) · CIFAR-10, 20% keep ratio | accuracy: +2.8 | Helped |
| replace uniform sampling with adaptive sampling using bin importance | DQ (ADQ) · CIFAR-10, 30% keep ratio | accuracy: +3.3 | Helped |
| add RS (ablation) | DQ · CIFAR-10, 10%, 20%, and 30% keep ratios | accuracy: 84.1 -> 85.1; 87.6 -> 88.8; 91.0 -> 93.0 | Helped |
| add DS (ablation) | DQ · CIFAR-10, 10%, 20%, and 30% keep ratios | accuracy: 84.1 -> 85.3; 87.6 -> 88.9; 91.0 -> 93.1 | Helped |
| add IS (RS+DS) (ablation) | DQ · CIFAR-10, 10%, 20%, and 30% keep ratios | accuracy: 84.1 -> 86.2; 87.6 -> 90.4; 91.0 -> 94.2 | Helped |
| add RS, DS, or IS (RS+DS) (ablation) | DQ · CIFAR-10, 100% keep ratio | accuracy: 95.6 -> 95.6 | No clear effect |
| vary alpha from 0 toward 1 to weight bin importance in sampling (ablation) | ADQ · CIFAR-10 | accuracy | Mixed |
Surprising findings
- The paper reports that DQ and ADQ are near-optimal only under an ideal condition of equal bin importance, and that bin importance first rises then falls over the generation process.
Limitations
- Gains are modest (about 3% average over DQ) and depend on tuning the weighting coefficient alpha per architecture.
- Evaluation is primarily on CIFAR-10 and ImageNet-1K with image classification as the only proxy task.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K Tiny-ImageNet
Benchmarks: CIFAR-10 ImageNet-1K
Metrics: test accuracy average accuracy across architectures GPU hours error bars
Baselines: DM DQ GraphCut
Builds on: Dataset Quantization GraphCut Dataset Distillation Masked Autoencoder
Closest papers by meaning
- Post Training Quantization for Efficient Dataset Condensation (AAAI 2026 · similarity 0.89)
- Enhancing Diversity for Data-free Quantization (CVPR 2025 · similarity 0.87)
- Rethinking Dataset Distillation for Classification: Do Distilled Sets Outperform Coresets? (NeurIPS 2026 · similarity 0.86)
- Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty (ICML 2025 · similarity 0.86)
- Optimal and Approximate Adaptive Stochastic Quantization (NeurIPS 2024 · similarity 0.85)
- Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.