Research map / Learning theory and algorithms / Prediction and decision losses

Learning-Efficient Yet Generalizable Collaborative Filtering for Item Recommendation

ICML 2024 · Learning theory and algorithms › Prediction and decision losses · approach: Ranking-based loss optimization

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RG2, a squared loss derived by Taylor-expanding the DCG-consistent softmax loss, proves its DCG consistency, and optimizes matrix factorization with closed-form ALS updates over all items.

Problem

The weighted squared loss used in iALS/WRMF achieves strong ranking performance, but it lacks a clear theoretical connection to ranking objectives like DCG, and softmax loss (which is DCG-consistent) cannot easily use closed-form ALS optimization.

Approach

Taylor-expand the softmax loss at zero to obtain a squared-form surrogate (RG2) that upper-bounds softmax, prove its DCG-consistency, and use it with MF optimized by ALS with closed-form updates over all items, removing the weight hyperparameter alpha.

Key results

• On MovieLens, RG2 reaches NDCG@10 0.2963 and MRR@10 0.4723, versus Softmax 0.2849 / 0.4487 and WRMF 0.2797 / 0.4475.
• On Electronics and Steam, RG2 is roughly on par with Softmax (e.g., Steam NDCG@10 0.0575 vs 0.0579), with RG2 converging faster than Softmax in wall-clock time (Figure 1).
• RG2 and WRMF both converge faster than SGD-trained Softmax thanks to closed-form ALS updates.

Measured effects reported in the paper

Surprising findings

• RG2 slightly surpasses Softmax on MovieLens despite being a squared-loss approximation of it.
• Sparsemax performs much worse than other losses, attributed to underfitting inactive users and cold items.

Limitations

• Consistency and generalization bounds are theoretical; the generalization bound is loose (exponent-dependent rates), and the paper notes the hypothesis space need not map into [0,1].
• Experiments are limited to MF with a single optimizer setup on three datasets; graph-based results are only in the appendix.
• Performance on Electronics and Steam is slightly below Softmax in some metrics.

Experimental setup

Datasets: MovieLens-10M Amazon Electronics Steam

Benchmarks: MovieLens-10M Amazon Electronics Steam

Metrics: MRR@10 NDCG@10

Baselines: BPR BCE Sampled Softmax Sparsemax UIB SML WRMF Softmax

Builds on: iALS / WRMF softmax loss DCG-consistency (Bruch et al. 2019) Srebro et al. pseudo-dimension bound

Closest papers by meaning

• PSL: Rethinking and Improving Softmax Loss from Pairwise Perspective for Recommendation (NeurIPS 2024 · similarity 0.88)
• From Pairwise to Ranking: Climbing the Ladder to Ideal Collaborative Filtering with Pseudo-Ranking (AAAI 2025 · similarity 0.88)
• Advancing Loss Functions in Recommender Systems: A Comparative Study with a Rényi Divergence-Based Solution (AAAI 2025 · similarity 0.88)
• Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering (NeurIPS 2022 · similarity 0.87)
• Recommendations with Sparse Comparison Data: Provably Fast Convergence for Nonconvex Matrix Factorization (ICML 2025 · similarity 0.87)
• Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation (NeurIPS 2025 · similarity 0.85)