Research map / Label-efficient and robust learning / Knowledge distillation

SHARP-Distill: A 68× Faster Recommender System with Hypergraph Neural Networks and Language Models

ICML 2025 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SHARP-Distill, a teacher-student distillation framework where a DeBERTa-augmented dual-HGNN teacher transfers structural and positional knowledge via contrastive learning to a lightweight single-layer CompactGCN student for faster recommendation inference.

Problem

GNN and hypergraph neural network recommenders capture high-order user-item relations but incur high inference cost, and existing soft-label distillation transfers structural knowledge poorly; sparse interactions also limit accuracy.

Approach

Distill a dual-HGNN + DeBERTa review-aware teacher into a shallow CompactGCN student using both cross-modal/intra-domain contrastive alignment and a hypergraph positional-encoding similarity, combined with soft-label distillation and embedding interpolation.

Key results

• SHARP-Distill reports Yelp P@10 of 3.88 vs 3.52 for the best baseline LightGCN-family entry (HCCF 3.52), and Sports P@10 of 4.27 vs 3.56 for HGAtt.
• Inference on Amazon CDs at 136,701 nodes: 9.77 ms for SHARP-Distill vs 395.45 ms LightGCN (40x) and 668.23 ms HGNN (68x).
• Ablation on Yelp shows removing DeBERTa drops P@10 by 20.13% and removing contrastive learning drops it by 32.35%.

Measured effects reported in the paper

Surprising findings

• Reported Table 1 values show HCCF as the strongest baseline on Yelp P@10 at 3.52, while the paper's own Table 2 reports HR values from a different scale, making cross-table comparisons inconsistent.
• The Amazon Sports N@10 for SHARP-Distill (3.24) is nearly three times the best baseline (1.74), an unusually large gap for this task.

Limitations

• Speedup claims rely on inference timings that are inconsistent across tables (e.g., Table 3 LightGCN/HGNN timings vs Figure 4 timings), and hardware is not described.
• Evaluation covers only five datasets with a single-seed-averaged protocol and no statistical significance tests beyond standard deviations.
• Teacher is expensive to train (HGNN + DeBERTa, 145M params), and the framework depends on review text being available.

Experimental setup

Datasets: Yelp Tips Amazon CDs Amazon Cellphones Amazon Beauty Amazon Sports

Benchmarks: Yelp Tips Amazon CDs Amazon Cellphones Amazon Beauty Amazon Sports

Metrics: Precision@10 Recall@10 NDCG@10 F1 Hit Ratio@10/20/50 inference time (ms) CKA

Baselines: LightGCN HCCF HGAtt GCGM SAID POD KRD LightHGNN

Builds on: HGNN DeBERTa LightGCN GLNN KRD LightHGNN

Closest papers by meaning

• Semantic Enhanced Heterogeneous Hypergraph Network for Collaborative Filtering (AAAI 2025 · similarity 0.89)
• Active Large Language Model-Based Knowledge Distillation for Session-Based Recommendation (AAAI 2025 · similarity 0.86)
• ContextGNN: Beyond Two-Tower Recommendation Systems (ICLR 2025 · similarity 0.86)
• Tail-STEAK: Improve Friend Recommendation for Tail Users via Self-Training Enhanced Knowledge Distillation (AAAI 2024 · similarity 0.86)
• Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering (ICML 2025 · similarity 0.86)
• DivGCL: A Graph Contrastive Learning Model for Diverse Recommendation (AAAI 2025 · similarity 0.85)