Research map / Graph learning / Graph self-supervised learning

StructComp: Substituting propagation with Structural Compression in Training Graph Contrastive Learning

ICLR 2024 · Graph learning › Graph self-supervised learning · approach: Contrastive learning with negatives and views · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes StructComp, which replaces message passing in graph contrastive learning training with METIS-based node compression so the encoder trains as an MLP on compressed nodes, then uses the full GNN at inference.

Problem

Graph contrastive learning is hard to scale because message passing over many nodes and the quadratic number of contrastive sample pairs make training expensive, and sampling or decoupling methods from supervised GNN training do not transfer well to GCL.

Approach

Approximate the diffusion matrix with a sparse low-rank graph partition matrix (via METIS), compress each cluster to its mean feature, and train an MLP encoder with the unchanged GCL loss on compressed nodes and compressed-graph positive/negative pairs; at inference, transfer weights to the GNN encoder. Multi-view variant adds DropMember augmentation on cluster members.

Key results

• On small datasets, StructComp improves GRACE by +2.6 average accuracy and CCA-SSG by +1.6, while SCE and COLES gain +0.4 and +0.2.
• Memory reduced by up to 57.1x and training sped up by up to 21.1x on small datasets relative to full graph training.
• On Ogbn-Products, StructComp-trained models reach 75.2-75.8% accuracy versus 74.1-75.4% for Cluster-GCN and GraphSAINT, with lower time and memory.

Measured effects reported in the paper

Surprising findings

• Compressing graphs with only a few thousand clusters (e.g., 5000 for Papers100M) still matches or exceeds full-graph accuracy, suggesting the message-passing-free MLP training transfers well to GNN inference.

Limitations

• Theoretical equivalence is proven mainly for linear one-layer message passing and Erdős-Rényi random graphs, with the nonlinear extension sketched in the appendix.
• Requires a graph partitioning preprocessing step (METIS) and a key hyperparameter (number of clusters) that must be tuned per dataset.

Experimental setup

Datasets: Cora Citeseer Pubmed Amazon-Photo Amazon-Computers Ogbn-Arxiv Ogbn-Products Ogbn-Papers100M

Benchmarks: Cora Citeseer Pubmed Amazon-Photo Amazon-Computers Ogbn-Arxiv Ogbn-Products Ogbn-Papers100M

Metrics: classification accuracy training time per epoch memory usage

Baselines: Cluster-GCN GraphSAINT GraphZoom full graph training GGD BlockGCL SP-GCL Contrast-Reg

Builds on: GRACE CCA-SSG SCE COLES METIS

Closest papers by meaning

• ReGCL: Rethinking Message Passing in Graph Contrastive Learning (AAAI 2024 · similarity 0.91)
• Beyond Homophily: Graph Contrastive Learning with Macro-Micro Message Passing (AAAI 2025 · similarity 0.90)
• S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning (ICML 2024 · similarity 0.90)
• Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling (AAAI 2025 · similarity 0.89)
• Exploitation of a Latent Mechanism in Graph Contrastive Learning: Representation Scattering (NeurIPS 2024 · similarity 0.89)
• Graph Contrastive Invariant Learning from the Causal Perspective (AAAI 2024 · similarity 0.88)