Research map / Label-efficient and robust learning / Knowledge distillation
Tail-STEAK: Improve Friend Recommendation for Tail Users via Self-Training Enhanced Knowledge Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Tail-STEAK, a model-agnostic two-stage self-training plus self-knowledge distillation framework that improves GNN-based friend recommendation for low-degree (tail) users.
Problem
Degree-related bias in GNN-based friend recommendation: tail users with few links suffer from label sparsity and neighborhood sparsity, leading to much worse ranking performance than head users.
Approach
Two-stage self-training (head users first, then pseudo links predicted for tail users) combined with data-augmentation-based self-knowledge distillation that uses mutual-information (InfoNCE) maximization between head and synthesized tail views (link dropout and ID embedding noise) instead of reconstruction loss.
Key results
- On Deezer with SHGN, Tail-STEAK-full improves tail-user NDCG@10 by 14.22% and MRR by 17.33% over the base model (tail users have degree ≤ median).
- On Last.FM with LightGCN, tail-user NDCG@10 improves by 48.15% and MRR by 58.37% over the base model.
- Head-user performance of Tail-STEAK drops slightly relative to the best baselines but remains competitive, and it outperforms Tail-GNN and LFT on head users in most cases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Tail-STEAKfull | SHGN · Deezer tail users (≤ T) | NDCG@10: 0.3108 -> 0.3550 | Helped |
| apply Tail-STEAKfull | LightGCN · Deezer tail users (≤ T) | NDCG@10: 0.2499 -> 0.3139 | Helped |
| apply Tail-STEAKfull | SHGN · Last.FM tail users (≤ T) | NDCG@10: 0.4214 -> 0.5409 | Helped |
| apply Tail-STEAKfull | LightGCN · Last.FM tail users (≤ T) | NDCG@10: 0.3328 -> 0.4199 | Helped |
| add ID embedding disturbance (ablation) | Tail-STEAK, SHGN · Deezer tail users (≤ T) | NDCG@10: 0.3258 -> 0.3550 | Helped |
| add ID embedding disturbance (ablation) | Tail-STEAK, LightGCN · Deezer tail users (≤ T) | NDCG@10: 0.3225 -> 0.3139 | Hurt |
| add second-stage tail-based knowledge distillation (ablation) | Tail-STEAK, SHGN · Deezer tail users (≤ T) | NDCG@10: 0.3685 -> 0.3550 | Hurt |
| add second-stage tail-based knowledge distillation (ablation) | Tail-STEAK, LightGCN · Deezer tail users (≤ T) | NDCG@10: 0.2585 -> 0.3139 | Helped |
| add second-stage knowledge-distillation terms (ablation) | Tail-STEAK, SHGN · Deezer tail users (≤ T) | NDCG@10: 0.3628 -> 0.3550 | No clear effect |
| add second-stage knowledge-distillation terms (ablation) | Tail-STEAK, LightGCN · Deezer tail users (≤ T) | NDCG@10: 0.2575 -> 0.3139 | Helped |
| add the second training stage (ablation) | Tail-STEAK, SHGN · Deezer tail users (≤ T) | NDCG@10: 0.3656 -> 0.3550 | Hurt |
| add the second training stage (ablation) | Tail-STEAK, LightGCN · Deezer tail users (≤ T) | NDCG@10: 0.2481 -> 0.3139 | Helped |
Surprising findings
- Pseudo-link prediction and tail-based distillation are harmful or of limited benefit for SHGN, while head-based distillation is key; the components' effects depend heavily on base architecture.
- ID embedding disturbance helps SHGN but hurts LightGCN on tail users.
Limitations
- Gains for tail users come at a cost of head-user performance, which drops relative to the best baselines.
- Effects of components are inconsistent across base models (e.g., SHGN vs LightGCN), so the method is not uniformly beneficial.
- Evaluated only on two social network datasets and two base GNNs.
Experimental setup
Datasets: Deezer Last.FM
Benchmarks: Deezer Last.FM
Metrics: NDCG@10 MRR
Baselines: DGI GRACE MvGRL SGL NCL SimGCL LFT MoE Tail-GNN SSNet
Builds on: Simple-HGN LightGCN BPR self-knowledge distillation via data augmentation (DDGSD)
Closest papers by meaning
- SHARP-Distill: A 68× Faster Recommender System with Hypergraph Neural Networks and Language Models (ICML 2025 · similarity 0.86)
- Disentangled Modeling of Preferences and Social Influence for Group Recommendation (AAAI 2025 · similarity 0.86)
- ContextGNN: Beyond Two-Tower Recommendation Systems (ICLR 2025 · similarity 0.86)
- DOGE: LLMs-Enhanced Hyper-Knowledge Graph Recommender for Multimodal Recommendation (AAAI 2025 · similarity 0.85)
- Robust Graph Based Social Recommendation Through Contrastive Multi-View Learning (AAAI 2025 · similarity 0.85)
- Semantic Enhanced Heterogeneous Hypergraph Network for Collaborative Filtering (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.