Research map / Label-efficient and robust learning / Recognition and long-tail
ConR: Contrastive Regularizer for Deep Imbalanced Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConR, a contrastive regularizer adapted from infoNCE with multi-positive pairs and label-similarity-weighted negative pushing, to improve deep imbalanced regression across age, depth, and gaze estimation benchmarks.
Problem
Deep imbalanced regression: features of minority-label samples collapse into majority-label neighbours, and existing imbalanced methods are designed for categorical labels or assume label ordering.
Approach
A continuous, label-aware contrastive regularizer: anchors are chosen where label and prediction similarities disagree, positives are samples with similar labels, and negatives (similar predictions but dissimilar labels) are pushed away with weights proportional to a density-based pushing power and label dissimilarity. It is added to a regression loss and works with uni- and multi-dimensional labe
Key results
- On AgeDB-DIR, LDS+FDS+RankSim+ConR reaches 6.81 MAE overall and 9.21 few-shot MAE, the best reported.
- On IMDB-WIKI-DIR, FDS+ConR achieves 7.29 overall MAE and RankSim+FDS+ConR 21.32 few-shot MAE.
- On NYUD2-DIR, Balanced MSE (BNI)+ConR reaches 1.265 RMSE overall versus 1.283 without ConR; ConR training time is lower than FDS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add prediction-informed negative pair selection instead of selecting negatives only by label similarity (ablation) | ConR · AgeDB-DIR All | MAE: 7.48 -> 7.69 | Helped |
| Add pushing power to negative pairs (ablation) | ConR · AgeDB-DIR All | MAE: 7.51 -> 7.48 | Helped |
| Make pushing weights proportional to label similarity (ablation) | ConR · AgeDB-DIR All | MAE: 7.54 -> 7.48 | Helped |
| Make pushing powers proportional to instance weights (ablation) | ConR · AgeDB-DIR All | MAE: 7.52 -> 7.48 | Helped |
| Add prediction-informed negative pair selection instead of selecting negatives only by label similarity (ablation) | ConR · IMDB-WIKI-DIR All | MAE: 8.10 -> 7.84 | Helped |
| Add pushing power to negative pairs (ablation) | ConR · IMDB-WIKI-DIR All | MAE: 7.79 -> 7.84 | No clear effect |
| Make pushing weights proportional to label similarity (ablation) | ConR · IMDB-WIKI-DIR All | MAE: 7.83 -> 7.84 | No clear effect |
| Make pushing powers proportional to instance weights (ablation) | ConR · IMDB-WIKI-DIR All | MAE: 7.76 -> 7.84 | No clear effect |
| Add prediction-informed negative pair selection instead of selecting negatives only by label similarity (ablation) | ConR · NYUD2-DIR All | RMSE: 1.518 -> 1.304 | Helped |
| Add pushing power to negative pairs (ablation) | ConR · NYUD2-DIR All | RMSE: 1.410 -> 1.304 | Helped |
| Make pushing weights proportional to label similarity (ablation) | ConR · NYUD2-DIR All | RMSE: 1.383 -> 1.304 | Helped |
| Make pushing powers proportional to instance weights (ablation) | ConR · NYUD2-DIR All | RMSE: 1.318 -> 1.304 | Helped |
Surprising findings
- Contrastive-ConR (label-similarity-only negatives, SupCon-style) performs better on many-shot but worse on few-shot, showing the bias toward majority samples.
- RankSim cannot be applied to depth or gaze estimation, since order relations are not meaningful there.
Limitations
- Gains on some shots are small or absent (e.g., Balanced MSE with ConR is worse on some shots of NYUD2-DIR and MPIIGaze-DIR many-shot), and hyperparameters (omega, alpha, beta, eta) are dataset-specific.
- Similarity thresholds and pushing-weight functions must be tuned per label space.
Experimental setup
Datasets: AgeDB-DIR IMDB-WIKI-DIR NYUD2-DIR MPIIGaze-DIR
Benchmarks: AgeDB-DIR IMDB-WIKI-DIR NYUD2-DIR MPIIGaze-DIR
Metrics: MAE GM RMSE delta_1 mean angular error
Baselines: LDS FDS RankSim Balanced MSE RRT Focal-R MoCo MoCo V2 VANILLA
Builds on: infoNCE SupCon LDS FDS RankSim Balanced MSE
Closest papers by meaning
- ACCon: Angle-Compensated Contrastive Regularizer for Deep Regression (AAAI 2025 · similarity 0.91)
- Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression (AAAI 2025 · similarity 0.89)
- BaCon: Boosting Imbalanced Semi-supervised Learning via Balanced Feature-Level Contrastive Learning (AAAI 2024 · similarity 0.88)
- Gradient Aligned Regression via Pairwise Losses (ICML 2025 · similarity 0.87)
- Toward Robustness in Multi-Label Classification: A Data Augmentation Strategy against Imbalance and Noise (AAAI 2024 · similarity 0.86)
- PROTOCOL: Partial Optimal Transport-enhanced Contrastive Learning for Imbalanced Multi-view Clustering (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.