Research map / Label-efficient and robust learning / Knowledge distillation

DistillHGNN: A Knowledge Distillation Approach for High-Speed Hypergraph Neural Networks

ICLR 2025 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DistillHGNN, a teacher-student knowledge distillation framework that compresses hypergraph neural networks into fast, memory-efficient MLP students while preserving accuracy.

Problem

Hypergraph neural networks are accurate but slow and memory-heavy at inference, limiting their practical use.

Approach

A teacher made of an HGNN plus an MLP produces soft labels and transfers structural and high-order relational knowledge to a lightweight student via knowledge distillation, enabling fast inference without the hypergraph structure at test time.

Experimental setup

Metrics: inference speed memory efficiency accuracy

Builds on: knowledge distillation

Closest papers by meaning

• LightHGNN: Distilling Hypergraph Neural Networks into MLPs for 100x Faster Inference (ICLR 2024 · similarity 0.92)
• VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs (ICLR 2024 · similarity 0.88)
• Training-Free Message Passing for Learning on Hypergraphs (ICLR 2025 · similarity 0.88)
• K-hop Hypergraph Neural Network: A Comprehensive Aggregation Approach (AAAI 2025 · similarity 0.88)
• Deep Hypergraph Neural Networks with Tight Framelets (AAAI 2025 · similarity 0.88)
• Large Language Model Meets Graph Neural Network in Knowledge Distillation (AAAI 2025 · similarity 0.87)