Research map / Label-efficient and robust learning / Recognition and long-tail
Hyperbolic Dataset Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HDD, which embeds frozen-encoder features into the Lorentz hyperboloid and matches original and synthetic sets by the hyperbolic distance between their centroids, plugging into existing distribution-matching dataset distillation methods.
Problem
Existing distribution matching (DM) dataset distillation methods work in Euclidean/Hilbert space and treat samples as i.i.d., ignoring the hierarchical, tree-like structure of dataset samples.
Approach
Map features into hyperbolic (Lorentz) space via the exponential map and match the Riemannian (Fréchet) centroids of real and synthetic sets using the geodesic distance, so that lower-level (prototype-like) samples get more weight in the centroid; the method is a drop-in loss for existing DM methods.
Key results
- IDM with HDD reaches 47.0% on CIFAR-10 at IPC=1 (+1.8 over IDM) and 61.3% at IPC=10 (+4.0 over IDM).
- DM with HDD improves SVHN IPC=1 accuracy from 21.9% to 25.0%.
- Hierarchical pruning keeps performance with 20% of CIFAR-10 data for DM with HDD (50.2% at 80% pruning vs 50.3% unpruned, reported at pruning ratio 80%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HDD to IDM | IDM · Fashion-MNIST, SVHN, CIFAR-10, and CIFAR-100 (IPC 1, 10, and 50) | accuracy: 78.5% (+1.1%), 67.8% (+2.5%), 47.0% (+1.8%), 25.3% (+3.2%) at IPC = 1; 61.3% (+4.0%) on CIFAR-10 at IPC = 10; gains of 2 | Helped |
| Add HDD to DM | DM · SVHN (IPC=1 and IPC=10), CIFAR-10 (IPC=1) | accuracy: 3.1% (IPC = 1) and 2.3% (IPC = 10); 2.3% (IPC = 1) | Helped |
| Add HDD to Dance | Dance · CIFAR-10, CIFAR-100, and ImageWoof | accuracy: 0.8% on CIFAR-10 IPC = 50; 1.5% on CIFAR-100 IPC = 1; 0.5% on ImageWoof IPC = 1 and 0.4% on ImageWoof IPC = 10 | Mixed |
| Add HDD to DSDM | DSDM · CIFAR-10 (IPC=1, 10, and 50) | accuracy: 2.6% (IPC = 1); 0.8% (IPC = 10) | Helped |
| Retain only 20% of the original CIFAR-10 data for hierarchical alignment (ablation) | DM with HDD and IDM with HDD · CIFAR-10 (IPC=10) | accuracy: 50.2$\pm$0.2 (DM with HDD, 80% pruning) vs 50.3$\pm$0.3 (0%); 60.3$\pm$0.3 (IDM with HDD, 80% pruning) vs 61.3$\pm$0.1 ( | Mixed |
| Increase hyperbolic curvature magnitude from |K|=1/3 to |K|=5 (ablation) | DM with HDD · CIFAR-10 (IPC=1, 10, and 50) | accuracy: 27.0$\pm$0.2 to 28.6$\pm$0.2; 49.6$\pm$0.3 to 50.0$\pm$0.2; 63.0$\pm$0.3 to 62.7$\pm$0.1 | No clear effect |
Surprising findings
- Pruning to 20% of the data (keeping lower-level samples) preserves distillation accuracy and reduces accuracy fluctuation, suggesting higher-level samples contribute little to the centroid.
Limitations
- Curvature and gradient scaling factor lambda must be tuned per dataset and IPC, with the distance otherwise very small near the origin.
- Tangent-space approximation justifying the per-sample weighting is only approximate near the origin.
- Relies on randomly initialized or frozen encoder features; the hierarchy is assumed rather than verified.
Experimental setup
Datasets: Fashion-MNIST SVHN CIFAR-10 CIFAR-100 Tiny ImageNet ImageWoof
Benchmarks: Fashion-MNIST SVHN CIFAR-10 CIFAR-100 Tiny ImageNet ImageWoof
Metrics: test accuracy (%) runtime GPU memory
Baselines: DM IDM DC DSA CAFE DCC DataDAM G-VBSM DATM RDED D4M IID
Builds on: distribution matching IDM Dance DSA
Closest papers by meaning
- Large Scale Dataset Distillation with Domain Shift (ICML 2024 · similarity 0.87)
- OPTICAL: Leveraging Optimal Transport for Contribution Allocation in Dataset Distillation (CVPR 2025 · similarity 0.87)
- Dataset Distillation with Neural Characteristic Function: A Minmax Perspective (CVPR 2025 · similarity 0.87)
- Taming Diffusion for Dataset Distillation with High Representativeness (ICML 2025 · similarity 0.87)
- Dataset Distillation of 3D Point Clouds via Distribution Matching (NeurIPS 2025 · similarity 0.87)
- Fast Hyperboloid Decision Tree Algorithms (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.