Research map / Optimization / Implicit bias and generalization

Constrained Factorization with Diagonal Scaling: Rank-Revealing Training and Pruning

NeurIPS 2026 · Optimization › Implicit bias and generalization · approach: Implicit regularization of SGD

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes constrained matrix factorization with explicit diagonal scaling (X ≈ UDU^T, U bounded) for rank-revealing training and pruning of PSD matrices.

Problem

Gradient descent on matrix factorization shows implicit low-rank bias, but iterates can grow unbounded; the paper seeks a principled, rank-revealing formulation for PSD matrix recovery.

Approach

Separate bounded (constrained) outer factors from explicit diagonal scale parameters, so that the diagonal entries directly reveal rank and can be used for pruning, rather than relying on implicit bias of unconstrained factorization.

Closest papers by meaning

• On the Crucial Role of Initialization for Matrix Factorization (ICLR 2025 · similarity 0.89)
• Provable Acceleration of Nesterov's Accelerated Gradient for Asymmetric Matrix Factorization and Linear Neural Networks (NeurIPS 2024 · similarity 0.89)
• In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting (AAAI 2025 · similarity 0.88)
• Two Sparse Matrices are Better than One: Sparsifying Neural Networks with Double Sparse Factorization (ICLR 2025 · similarity 0.88)
• Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries (ICLR 2025 · similarity 0.88)
• Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion (NeurIPS 2024 · similarity 0.88)