Research map / Learning theory and algorithms / Matrix and tensor methods
Structured low-rank learning
Applies low-rank structure to ranking, fairness and probability matrix problems with guarantees.
87 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Matrix and tensor methods, keyword cluster “matrices · matrix completion · rank matrix”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
12 papers in 2023–24 and 21 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 6.9% to 11.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Recognizing Emotions From Abstract Paintings Using Non-Linear Matrix Completion (CVPR 2016 · 107 citations)
- 2017 · Low-Rank Linear Cold-Start Recommendation from Social Data (AAAI 2017 · 72 citations)
- 2018 · Confidence-Aware Matrix Factorization for Recommender Systems (AAAI 2018 · 50 citations)
- 2019 · Holographic Factorization Machines for Recommendation (AAAI 2019 · 18 citations)
- 2020 · Fast Adaptively Weighted Matrix Factorization for Recommendation with Implicit Feedback (AAAI 2020 · 67 citations)
- 2021 · PSD Representations for Effective Probability Models (NeurIPS 2021 · 4 citations)
- 2022 · Moving Window Regression: A Novel Approach to Ordinal Regression (CVPR 2022 · 92 citations)
- 2023 · Efficient Top-K Feature Selection Using Coordinate Descent Method (AAAI 2023 · 15 citations)
- 2024 · HyperAttention: Long-context Attention in Near-Linear Time (ICLR 2024 · 9 citations)
- 2025 · Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner (NeurIPS 2025 · 1 citation)
- 2026 · Gaming Consensus: Coordinated Manipulation in Crowdsourced Fact-Checking (ICML 2026 · 0 citations)
Earliest papers here
- Recognizing Emotions From Abstract Paintings Using Non-Linear Matrix Completion (CVPR 2016 · 107 citations)
- Top-N Recommender System via Matrix Completion (AAAI 2016 · 98 citations)
- Multi-Stage Multi-Task Learning with Reduced Rank (AAAI 2016 · 79 citations)
What has been tried
72 papers here report 566 measured changes (a component added, removed or swapped, compared with a baseline): helped 69%, no clear effect 8%, hurt 19%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| replace 14 attention layers with HyperAttention | LongBench single-qa · task score | 80.63 -> 71.32 | Hurt | HyperAttention: Long-context Attention in Near-Linear Time (ICLR 2024 · 9 citations) |
| add RRR score computation | all tested data sets · performance | Helped | LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search (NeurIPS 2024 · 4 citations) | |
| Use stricter eigenbasis accuracy early (τ=0.01 for first 10%, then τ=0.8) rather than τ=0.8 throughout | Imagewoof ViT · final loss | ≈ 0.04 | Helped | Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner (NeurIPS 2025 · 1 citation) |
| remove decoder | FrcSub · ACC | 0.8267±0.0048 -> 0.7523±0.0118 | Hurt | An Autoencoder-Like Nonnegative Matrix Co-Factorization for Improved Student Cognitive Modeling (NeurIPS 2024 · 1 citation) |
| Use geometric operations on box embeddings rather than score multiplication | Last-FM, U ∩ A₁ ∩ ¬A₂ · Hit Rate@50 (%) | 60.3 -> 60.0 | No clear effect | A Geometric Approach to Personalized Recommendation with Set-Theoretic Constraints Using Box Embeddings (ICML 2025 · 0 citations) |
| Add the standardization step for estimated components | Dense and sparse graphons · estimation performance | Helped | Low-Rank Graphon Learning for Networks (NeurIPS 2025 · 0 citations) | |
| Average sampled-label gradients across the batch | Gauss–Newton matrix · expected approximation | No clear effect | A New Perspective on Shampoo's Preconditioner (ICLR 2025 · 0 citations) | |
| add orthogonal regularization | UTKFace · HV | 0.309 (0.002) -> 0.314 (0.001) | Helped | Efficient Pareto Manifold Learning with Low-Rank Structure (ICML 2024 · 0 citations) |
| Use non-warm-start initializations instead of a warm start | Low-dimensional simulation (n₁=200, n₂=300) and high-dimensional simulation (n₁=2000, n₂=3000) · normalized reconstruction error | No clear effect | Recommendations with Sparse Comparison Data: Provably Fast Convergence for Nonconvex Matrix Factorization (ICML 2025) | |
| Add regularization (μ) to COALA | Commonsense reasoning, 90% compression · accuracy | 57.6 (±0.7) -> 59.0 (±0.7) (boolQ); 60.9 (±1.0) -> 62.8 (±1.0) (PIQA); 53.2 (±1.3) -> 54.0 (±1.3) (WiNoG); 34.6 (±0.5) - | Helped | COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation (NeurIPS 2025) |
What reviewers will expect
Datasets and baselines named in the 18 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet · 4 of 18 papers
- MNIST · 2 of 18 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 33 papers; from OpenAlex and the papers' affiliations).
- Adobe · 4 papers
- University of Cambridge · 3 papers
- Carnegie Mellon University · 2 papers
- Google · 2 papers
- Georgia Institute of Technology · 2 papers
- Fudan University · 2 papers
- EPFL · 2 papers
- Columbia University · 2 papers
- Harvard University · 2 papers
- University of Michigan - Ann Arbor · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Low-rank approximation theory (142 papers)
- Matrix completion and factorization (118 papers)
- Randomized sketching for matrices (96 papers)
- Geometry of low-rank and SPD structure (71 papers)
Elsewhere in the topic
- Nonconvex low-rank and sparse optimization (112 papers)
- Sparse regression with guarantees (94 papers)
- High-dimensional estimation theory (89 papers)
- Signal recovery and phase retrieval (66 papers)
- Sparse and robust PCA (66 papers)
- Tensor network decompositions (45 papers)
- Low-rank tensor completion (45 papers)
- Randomized tensor decomposition algorithms (43 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).