Research map / Graph learning / Graph self-supervised learning

Graph-enhanced Optimizers for Structure-aware Recommendation Embedding Evolution

NeurIPS 2024 · Graph learning › Graph self-supervised learning · approach: Semi-supervised node representation learning · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SEvo, a structure-aware embedding update that smooths optimizer variations over an item adjacency graph via a rescaled Neumann series, integrated into AdamW (with moment correction) for sequential recommendation.

Problem

Standard embedding updates (gradient or optimizer variations) do not ensure that related nodes (items linked by interaction or prior knowledge) evolve similarly, and GNN modules used to inject graph structure add training and inference overhead.

Approach

Apply graph-regularized smoothing directly to the optimizer's embedding variation via a rescaled Neumann series transform that is both structure-aware and direction-aware (with convergence guarantees), plus a moment-estimate correction for AdamW under sparse gradients; no change to inference.

Key results

• SEvo-enhanced SASRec improves HR@1 by 12.3% on average across datasets (SEvo-enhanced backbones average gains 9%-23%), with GRU4Rec+SEvo averaging +23.4%.
• On large-scale Clothing, SASRec+SEvo raises HR@1 from 0.0071 to 0.0171 (+139.1%) and NDCG@10 by +89.1%.
• SEvo adds only a few seconds per epoch and does not change inference time, unlike SR-GNN/LESSR/MAERec which need heavy training and inference overhead.

Measured effects reported in the paper

Surprising findings

• The iterative (momentum-style) graph approximation used in prior work is not always direction-aware and fails to converge for beta above 0.5 on some graphs, unlike the Neumann series form.

Limitations

• Theoretical convergence analysis relies on mild assumptions; the paper acknowledges the bound is not tight and better rates need stronger assumptions.
• Adjacency is built from consecutive item co-occurrence counts, so similarity estimation quality limits gains; other prior-knowledge applications are only preliminary.
• Ongoing adjacency normalization and handling outdated historical information in dynamic graphs remain open challenges.

Experimental setup

Datasets: Amazon Beauty Amazon Toys Amazon Tools MovieLens-1M Amazon Electronics Amazon Clothing

Benchmarks: Beauty Toys Tools MovieLens-1M Electronics Clothing

Metrics: HR@1 HR@5 HR@10 NDCG@5 NDCG@10

Baselines: LightGCN SR-GNN LESSR MAERec MF-BPR GRU4Rec SASRec BERT4Rec STOSA

Builds on: AdamW Adam SASRec APPNP LightGCN

Closest papers by meaning

• Structure Aware Incremental Learning with Personalized Imitation Weights for Recommender Systems (AAAI 2023 · similarity 0.88)
• Rethinking Contrastive Learning for Graph Collaborative Filtering: Limitations and a Simple Remedy (ICML 2026 · similarity 0.87)
• On the Embedding Collapse when Scaling up Recommendation Models (ICML 2024 · similarity 0.86)
• Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering (ICML 2025 · similarity 0.86)
• ContextGNN: Beyond Two-Tower Recommendation Systems (ICLR 2025 · similarity 0.85)
• Semantic Enhanced Heterogeneous Hypergraph Network for Collaborative Filtering (AAAI 2025 · similarity 0.85)