Research map / Label-efficient and robust learning / Recognition and long-tail
Unsupervised Feature Learning with Emergent Data-Driven Prototypicality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HACK, an unsupervised hyperbolic feature learner that packs particles uniformly in the Poincaré ball and assigns images to them via Hungarian matching, so that distance to the origin encodes data-driven prototypicality.
Problem
Unsupervised discovery of data-driven prototypicality: mapping images to a feature space where visual similarity is preserved and the location (distance to origin) encodes how typical an image is within its dataset.
Approach
Perform unsupervised metric feature learning in the 2D Poincaré ball with predefined uniformly packed particles (sphere packing via repulsion loss) and batch-wise Hungarian assignment of images to particles, so typical images emerge near the origin and atypical ones near the boundary.
Key results
- Training on the most atypical 10% of CIFAR10 achieves 22.67% higher accuracy than on the most typical 10%.
- On MNIST, HACK with 10% atypical samples outperforms UHML by 10.2%.
- Removing the most atypical 1% of examples improves adversarial accuracy by 8.7% on MNIST and 7.3% on CIFAR10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with the most atypical 10% of images instead of the most typical 10% | HACK-based instance selection · CIFAR10 | accuracy: 22.67% higher | Helped |
| Incorporate 10% atypical samples based on feature norm | HACK · MNIST | accuracy: 10.2% | Helped |
| Exclude 1% atypical examples | HACK · MNIST | accuracy: 5.7% | Helped |
| Remove the most atypical 1% of examples | HACK-based instance selection · MNIST | adversarial accuracy: 8.7% | Helped |
| Remove the most atypical examples | HACK-based instance selection · CIFAR10 | adversarial accuracy: 7.3% | Helped |
Surprising findings
- Training on atypical examples yields higher test accuracy than training on typical examples, contrary to the intuition that prototypical data is easier to learn from.
Limitations
- Prototypicality evaluation relies on congealing-based proxy ground truth, with no standard benchmark for prototypicality.
- Hungarian assignment cost grows cubically with batch size.
- Results on USA10kF are described as less intuitive and organization unclear.
Experimental setup
Datasets: MNIST CIFAR10 USA10kF Congealed MNIST
Benchmarks: MNIST CIFAR10 USA10kF Congealed MNIST
Metrics: accuracy adversarial accuracy under FGSM hyperbolic norm vs K-NN density prototypicality discovery quality
Baselines: Holdout Retraining Model Confidence UHML
Builds on: Poincaré ball model congealing UHML Holdout Retraining Model Confidence
Closest papers by meaning
- Unsupervised Hyperbolic Metric Learning (CVPR 2021 · similarity 0.88)
- Hyperbolic Contrastive Learning for Visual Representations Beyond Objects (CVPR 2023 · similarity 0.87)
- Hyperbolic Image Embeddings (CVPR 2020 · similarity 0.86)
- Hyperbolic Feature Augmentation via Distribution Estimation and Infinite Sampling on Manifolds (NeurIPS 2022 · similarity 0.86)
- Hyperbolic Category Discovery (CVPR 2025 · similarity 0.85)
- Fully Hyperbolic Convolutional Neural Networks for Computer Vision (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.