Research map / Learning theory and algorithms / Kernels and regression theory

Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model

NeurIPS 2025 · Learning theory and algorithms › Kernels and regression theory · approach: Double descent and overfitting · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Extends neural collapse to cumulative link model ordinal regression via the unconstrained feature model, proving ordinal neural collapse (within-class mean collapse, one-dimensional alignment, and ordered latent variables) under fixed thresholds and validating it on tabular and UTKFace data.

Problem

Whether neural-collapse-like geometry emerges in deep ordinal regression models built on threshold/cumulative link models, where latent variables play the role of classification logits.

Approach

Combines the cumulative link model (logit/probit/clog-log) with the unconstrained feature model; shows under log-concave inverse link and fixed thresholds that features collapse to class means, class means align with the classifier, and optimal latent variables are ordered. In the small-regularization limit, latent variables sit at midpoints of neighbouring thresholds for symmetric links.

Key results

• On the ER dataset with the logit model, ONC2-1 and ONC2-2 rapidly approached zero, and ONC1 decreased steadily as training proceeded, indicating feature collapse onto a one-dimensional subspace aligned with the classifier.
• With fixed thresholds, the ONC3 indicator took a small value early and kept decreasing, whereas with learnable thresholds it converged to a non-zero value.
• On UTKFace with ResNet101, fixed thresholds markedly improved classification accuracy and minimum sensitivity over learnable thresholds, while learnable thresholds kept minimum sensitivity at zero.

Measured effects reported in the paper

Surprising findings

• With learnable thresholds, minimum sensitivity stayed at zero on UTKFace, meaning at least one class was completely ignored by the model.
• Fixed thresholds gave faster and more stable convergence than learnable thresholds.

Limitations

• Theory assumes fixed thresholds; ONC1-2 under learnable thresholds is only supported empirically.
• The phase-structure result relies on assumed continuity and monotonicity of the optimal norm w* rather than a full proof.
• The ONC3 indicator is only valid for fixed, evenly spaced thresholds.

Experimental setup

Datasets: ER LE SW CA WR UTKFace

Benchmarks: ER UTKFace

Metrics: NLL MAE accuracy QWK within-1 accuracy minimum sensitivity ONC1 ONC2-1 ONC2-2 ONC3

Baselines: learnable thresholds

Builds on: Unconstrained Feature Model Neural Collapse Cumulative Link Model Prekopa-Leindler inequality

Closest papers by meaning

• The Prevalence of Neural Collapse in Neural Multivariate Regression (NeurIPS 2024 · similarity 0.91)
• Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime (ICML 2025 · similarity 0.90)
• The Persistence of Neural Collapse Despite Low-Rank Bias (NeurIPS 2025 · similarity 0.89)
• Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (NeurIPS 2025 · similarity 0.89)
• Generalized Neural Collapse for a Large Number of Classes (ICML 2024 · similarity 0.89)
• Neural collapse vs. low-rank bias: Is deep neural collapse really optimal? (NeurIPS 2024 · similarity 0.89)