Research map / Graph learning / Graph self-supervised learning
SDMG: Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDMG, a self-supervised diffusion model for graph representation learning that adds low-frequency feature/topology encoders and a multi-scale smoothing loss to emphasize low-frequency reconstruction over full MSE reconstruction.
Problem
Generation-oriented MSE reconstruction objectives in diffusion models are misaligned with representation learning on graphs; reconstructing all detail (especially high-frequency noise) can hurt downstream node and graph classification.
Approach
Reconstruct only global low-frequency graph information via learnable low-frequency encoders (MLP for topology, GAT for node features) used as conditioning, and replace pointwise MSE with a multi-scale cosine-similarity smoothing loss over k-hop aggregated features.
Key results
- SDMG (w/ mask) reaches 94.7% on Photo and 91.6% on Computer node classification, versus 93.8% and 89.8% for DDM.
- On graph classification, SDMG is best on four of five datasets, e.g. 91.58% on MUTAG versus 90.15% for DDM.
- Ablation on Photo: removing both low-frequency encoders and MSS drops accuracy from 94.73% to 83.65%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SDMG instead of the strongest listed baseline | graph representation learning · Computer, Photo, IMDB-B, IMDB-M, COLLAB, MUTAG | accuracy: 90.4±0.2 vs 89.7±0.3; 94.1±0.2 vs 93.8±0.2; 76.03±0.53 vs 75.52±0.66; 52.5±0.42 vs 52.3±2.8; 82.23±0.35 vs 80.70±0.18; 9 | Helped |
| Use SDMG instead of the strongest listed baseline | graph representation learning · Cora, CiteSeer, PubMed, OGBN-Arxiv, PROTEINS | accuracy: 83.6±0.6 vs 84.2±0.4; 73.2±0.5 vs 73.6±0.2; 80.0±0.4 vs 82.0±0.2; 70.6±0.2 vs 72.1±0.1; 73.17±0.16 vs 76.2±2.8 | Mixed |
| Add feature masking | SDMG · node classification: Cora, CiteSeer, PubMed, OGBN-Arxiv, Computer, Photo | accuracy: 84.3±0.5 vs 83.6±0.6; 73.9±0.4 vs 73.2±0.5; 80.0±0.5 vs 80.0±0.4; 72.1±0.3 vs 70.6±0.2; 91.6±0.2 vs 90.4±0.2; 94.7±0.2 v | Mixed |
| Add low-frequency encoders and MSS instead of removing both (ablation) | SDMG · Photo, Computer, IMDB-B, MUTAG | accuracy: 83.65 -> 94.73; 76.42 -> 91.64; 74.05 -> 76.03; 90.15 -> 91.58 | Helped |
| Add low-frequency encoders instead of replacing them with MLPs (ablation) | SDMG · Photo, Computer, IMDB-B, MUTAG | accuracy: 87.42 -> 94.73; 85.86 -> 91.64; 74.53 -> 76.03; 90.53 -> 91.58 | Helped |
| Use MSS instead of MSE (ablation) | SDMG · Photo, Computer, IMDB-B, MUTAG | accuracy: 93.46 -> 94.73; 89.46 -> 91.64; 75.20 -> 76.03; 90.40 -> 91.58 | Helped |
Surprising findings
- Reconstructing only the lowest ~5-20% of graph Laplacian frequencies already gives strong representations, while full-spectrum reconstruction overfits high-frequency noise and degrades accuracy.
- Pure MSE training eventually reintroduces high-frequency details and causes accuracy to drop after about 100 epochs, even with low-frequency encoders.
Limitations
- Graph Fourier-based low-frequency truncation is computationally expensive for large graphs, motivating learned encoders.
- Frequency-adaptive objectives are left for future work; the fixed low-frequency focus may discard useful high-frequency signal (Figure 3b shows occasional benefit).
Experimental setup
Datasets: Cora CiteSeer PubMed ogbn-arxiv Photo Computer IMDB-B IMDB-M PROTEINS COLLAB MUTAG
Benchmarks: Cora CiteSeer PubMed ogbn-arxiv Computer Photo IMDB-B IMDB-M PROTEINS COLLAB MUTAG
Metrics: accuracy
Baselines: DDM GraphMAE GraphACL BGRL CCA-SSG MVGRL DGI GCN GAT GIN InfoGraph GraphCL
Builds on: DDM denoising diffusion probabilistic models GraphMAE Graph Attention Network
Closest papers by meaning
- Unifying Generation and Prediction on Graphs with Latent Graph Diffusion (NeurIPS 2024 · similarity 0.89)
- SBGD: Improving Graph Diffusion Generative Model via Stochastic Block Diffusion (ICML 2025 · similarity 0.88)
- Generating Graphs via Spectral Diffusion (ICLR 2025 · similarity 0.88)
- Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.88)
- Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generation (ICML 2025 · similarity 0.87)
- Graph Generation with Diffusion Mixture (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.