Research map / Label-efficient and robust learning / Recognition and long-tail
Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows theoretically, via neural collapse and a feature-spread (angular bound) argument, that the post-hoc multiplicative logit adjustment (MLA) approximates an optimal decision-boundary adjustment (1vs1adjuster), and validates this empirically on CIFAR10/100-LT, ImageNet-LT and Helena.
Problem
Long-tailed recognition: post-hoc logit adjustment is used heuristically, and the theoretical reason multiplicative logit adjustment (MLA) works is unexplained.
Approach
Derive optimal pairwise decision-boundary angles from neural-collapse-based angular concentration bounds that scale with class sample size, then show that MLA with logit scaling n_k^{-gamma} approximates these angles via a rational approximation of tan, while additive LA cannot approximate them globally.
Key results
- On CIFAR100-LT with ResNet34, MLA and 1vs1adjuster achieve comparable average accuracy across five seeds, with the decision-boundary angle difference between MLA and the optimal method globally small.
- Optimal MLA exponent gamma_x* is about 0.71 on CIFAR100-LT (ResNet34) and 0.20 on ImageNet-LT, consistent with theory of 0.5 under neural collapse and lower when training accuracy is not 100%.
- On CIFAR10-LT the approximation is weaker in boundary angles (few classes), yet MLA still matches 1vs1adjuster test accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ALA with MLA (post-hoc multiplicative logit adjustment) for ETF-classifier models | MLA vs ALA · CIFAR10-LT, CIFAR100-LT, ImageNet-LT, Helena | average test accuracy | Mixed |
| Use MLA (multiplicative scaling by n_k^-gamma) to approximate the NC-based 1vs1adjuster boundaries | MLA · CIFAR100-LT | decision boundary angle difference from 1vs1adjuster | Helped |
| Tune gamma for MLA/1vs1adjuster when training accuracy is 100% (NC occurs) | MLA and 1vs1adjuster hyperparameters · CIFAR100-LT | optimal gamma | No clear effect |
| Lower optimal gamma (below 0.5) when training accuracy is not near 100% (NC not fully realized) | MLA and 1vs1adjuster hyperparameters · ImageNet-LT and Helena | optimal gamma | No clear effect |
| Small number of classes (CIFAR10-LT) breaks the approximation: decision boundaries of MLA and ALA differ substantially from 1vs1adjuster, though accuracy remains comparable | MLA · CIFAR10-LT | decision boundary angle difference / test accuracy | Mixed |
Surprising findings
- MLA's approximation to the optimal boundary holds even when NC is not fully realized (ImageNet-LT, Helena), and when the number of classes is small (CIFAR10-LT) despite large boundary-angle differences.
- The optimal MLA exponent exceeds the theoretical 0.5 on CIFAR100-LT, and falls below it on ImageNet-LT and Helena.
Limitations
- Theory assumes neural collapse, ETF classifier and that features of each class collapse to their mean, which does not strictly hold in practice.
- Approximation is poor for small numbers of classes (CIFAR10-LT), where psi is not close to pi/2.
- No deep theoretical analysis of ALA is provided; ALA's theory does not align with the optimal boundary adjustment.
Experimental setup
Datasets: CIFAR10-LT CIFAR100-LT ImageNet-LT Helena
Benchmarks: CIFAR10-LT CIFAR100-LT ImageNet-LT Helena
Metrics: average test accuracy decision boundary angle difference
Baselines: additive logit adjustment (ALA) MLA 1vs1adjuster
Builds on: neural collapse logit adjustment ETF classifier Rademacher complexity
Closest papers by meaning
- Exploring Weight Balancing on Long-Tailed Recognition Problem (ICLR 2024 · similarity 0.86)
- Decoupled Optimisation for Long-Tailed Visual Recognition (AAAI 2024 · similarity 0.85)
- Robust Logit Adjustment for Learning with Long-Tailed Noisy Data (AAAI 2025 · similarity 0.85)
- Long-tailed Recognition with Model Rebalancing (NeurIPS 2025 · similarity 0.85)
- Difficulty-aware Balancing Margin Loss for Long-tailed Recognition (AAAI 2025 · similarity 0.85)
- MaxSup: Overcoming Representation Collapse in Label Smoothing (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.