Research map / Label-efficient and robust learning / Recognition and long-tail
Class-Attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Class-attribute Priors (CAP), which generates class-specific loss/post-hoc logit-adjustment hyperparameters from class attributes via a low-dimensional linear map, improving balanced and fairness objectives on long-tailed and noisy CIFAR/ImageNet-LT.
Problem
Class heterogeneity (imbalance, label noise, difficulty, test-time weights) makes class-specific optimization hyperparameters costly (O(K) parameters) and unstable, especially for fairness objectives beyond balanced error.
Approach
Treat class-specific hyperparameters (loss weight, additive and multiplicative logit adjustments) as a function of class attributes (frequency, difficulty, noise, test weights) via a linear map over a feature dictionary, so that only O(1) meta-parameters are tuned on validation data for K classes.
Key results
- CAP bilevel with all attributes reaches 20.22% balanced error on CIFAR10-LT, 56.38% on CIFAR100-LT and 49.31% on ImageNet-LT, versus 21.15/56.70/50.91 for AutoBalance.
- On CIFAR100-LT post-hoc fairness objectives, CAP improves quantile error at a=0.2 to -18.34 vs -12.79 for plain post-hoc relative to the pre-trained model.
- Using frequency plus difficulty attributes on noisy CIFAR10-LT lowers balanced error to 38.36% vs 40.04% for AutoBalance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CAP with all attributes instead of AutoBalance | Bilevel loss design · CIFAR10-LT | balanced error: 21.15 -> 20.22 | Helped |
| Use CAP with all attributes instead of AutoBalance | Bilevel loss design · CIFAR100-LT | balanced error: 56.70 -> 56.38 | Helped |
| Use CAP with all attributes instead of AutoBalance | Bilevel loss design · ImageNet-LT | balanced error: 50.91 -> 49.31 | Helped |
| Optimize post-hoc adjustment with CAP instead of LA post-hoc | CIFAR100-LT post-hoc · CIFAR100-LT | standard deviation difference: -8.72($\pm$0.66) -> -13.92($\pm$0.24) | Helped |
| Combine frequency and difficulty attributes rather than use frequency alone (ablation) | CAP Bilevel · CIFAR100-LT | balanced error: 56.64 -> 56.38 | Helped |
| Combine frequency and difficulty attributes rather than use frequency alone (ablation) | CAP Bilevel · ImageNet-LT | balanced error: 50.82 -> 49.31 | Helped |
| Combine frequency and difficulty attributes rather than use frequency alone (ablation) | CAP Bilevel · CIFAR10-LT+Noise | balanced error: 39.91 -> 38.36 | Helped |
| Use difficulty rather than frequency attribute only (ablation) | CAP Bilevel · CIFAR100-LT | standard deviation of class errors: 27.13($\pm$0.35) -> 17.62($\pm$0.65) | Helped |
| Optimize additive and multiplicative adjustment together rather than additive adjustment only (ablation) | CAP Post-hoc · CIFAR10-LT | balanced error: 20.65($\pm$0.36) -> 20.32($\pm$0.64) | Hurt |
| Optimize multiplicative adjustment only rather than additive adjustment only (ablation) | CAP Post-hoc · CIFAR10-LT | balanced error: 20.65($\pm$0.36) -> 21.06($\pm$0.43) | Hurt |
| Optimize multiplicative adjustment only rather than additive adjustment only (ablation) | CAP Post-hoc · CIFAR100-LT | balanced error: 57.15($\pm$0.30) -> 57.26($\pm$0.35) | Hurt |
Surprising findings
- Optimizing multiplicative logit adjustment (Delta) alone or jointly with additive l degrades balanced error in post-hoc tuning, with test loss decreasing while balanced error increases.
Limitations
- Shares one meta-strategy across classes, so it cannot personalize per class given infinite data.
- Bilevel search on a small imbalanced 20% validation split is prone to overfitting for tail classes.
- Evaluated only on CIFAR-LT and ImageNet-LT with ResNet backbones.
Experimental setup
Datasets: CIFAR10-LT CIFAR100-LT ImageNet-LT
Benchmarks: CIFAR10-LT CIFAR100-LT ImageNet-LT
Metrics: balanced error standard deviation of class-conditional errors quantile class error CVaR weighted error
Baselines: Cross entropy Logit adjustment (LA) CDT AutoBalance Post-hoc LA Plain post-hoc
Builds on: logit adjustment loss VS loss AutoBalance CDT
Closest papers by meaning
- Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance (ICML 2024 · similarity 0.88)
- Leveraging Uncertainty Estimates To Improve Classifier Performance (ICLR 2024 · similarity 0.86)
- Classes Are Not Equal: An Empirical Study on Image Recognition Fairness (CVPR 2024 · similarity 0.86)
- Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data (ICML 2025 · similarity 0.86)
- FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA (NeurIPS 2025 · similarity 0.86)
- Learning under Localized Minority Imbalance (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.