Research map / Graph learning / GNN architectures
Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Simi-Mailbox, a post-hoc GNN calibration method that clusters nodes by neighborhood prediction similarity and confidence and learns one temperature per cluster.
Problem
Post-hoc calibration of GNN node classification confidence; existing neighborhood-similarity-based node-wise temperature methods (CaGCN, GATS) assume a uniform trend and miscalibrate nodes with similar neighborhood similarity but different confidence.
Approach
Group nodes by both neighborhood representational similarity (mailbox value from sigmoid of logit dot products) and normalized confidence using KMeans, then learn a group-specific temperature per cluster, optimized to match per-cluster average confidence to accuracy.
Key results
- Simi-Mailbox achieves lowest ECE in 15 of 16 small/medium settings, e.g., Cora GCN ECE 1.97% vs GATS 3.55%.
- On large-scale graphs, ECE below 1% in all settings; Reddit SAGE ECE 0.73% vs uncalibrated 11.30%.
- Calibration time reduced versus GATS, e.g., Reddit SAGE 9.91s vs 192.01s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Simi-Mailbox instead of the strongest listed calibration baseline | GCN · Cora | ECE: 3.55 ± 1.28 -> 1.97 ± 0.44 | Helped |
| Use Simi-Mailbox instead of the strongest listed calibration baseline | GCN · Citeseer | ECE: 4.49 ± 1.53 -> 2.66 ± 0.53 | Helped |
| Use Simi-Mailbox instead of the strongest listed calibration baseline | GCN · Pubmed | ECE: 0.95 ± 0.32 -> 0.75 ± 0.15 | Helped |
| Use Simi-Mailbox instead of the strongest listed calibration baseline | GCN · Computers | ECE: 1.70 ± 0.53 -> 1.02 ± 0.26 | Helped |
| Use Simi-Mailbox instead of the strongest listed calibration baseline | GCN · CS | ECE: 0.90 ± 0.29 -> 0.58 ± 0.19 | Helped |
| Use Simi-Mailbox instead of GATS | GAT · Physics | ECE: 0.43 ± 0.16 -> 0.48 ± 0.22 | Hurt |
| Use Simi-Mailbox instead of CaGCN | GCN · Reddit | ECE: 1.86 ± 0.19 -> 0.35 ± 0.05 | Helped |
| Use Simi-Mailbox instead of CaGCN | SAGE · Reddit | ECE: 2.14 ± 0.35 -> 0.73 ± 0.15 | Helped |
| Replace min-max scaling with standard scaling (ablation) | Simi-Mailbox with GCN · Cora | ECE: 1.97 ± 0.44 -> 2.02 ± 0.54 | Hurt |
| Replace min-max scaling with standard scaling (ablation) | Simi-Mailbox with GCN · CoraFull | ECE: 3.46 ± 1.31 -> 3.53 ± 1.30 | Hurt |
| Replace min-max scaling with standard scaling (ablation) | Simi-Mailbox with GAT · Cora | ECE: 2.08 ± 0.45 -> 2.23 ± 0.41 | Hurt |
| Replace min-max scaling with standard scaling (ablation) | Simi-Mailbox with GAT · Computers | ECE: 0.95 ± 0.37 -> 0.94 ± 0.37 | Helped |
Surprising findings
- Calibration error varies substantially among nodes with similar neighborhood similarity but different confidence levels, contradicting the uniform trend assumed by CaGCN and GATS.
Limitations
- Relies on hand-chosen number of clusters N and regularization lambda; no theoretical foundation provided.
- Gains on some settings are marginal (e.g., Physics GAT where GATS is slightly better).
Experimental setup
Datasets: Cora Citeseer Pubmed CoraFull Coauthor CS Computers Photo Physics Arxiv Reddit
Benchmarks: Cora Citeseer Pubmed CoraFull Coauthor CS Computers Photo Physics Arxiv Reddit
Metrics: ECE class-wise ECE KDE-ECE Brier score NLL
Baselines: UnCal. TS VS ETS CaGCN GATS GC
Builds on: temperature scaling CaGCN GATS GCN GAT
Closest papers by meaning
- SimCalib: Graph Neural Network Calibration Based on Similarity between Nodes (AAAI 2024 · similarity 0.91)
- GETS: Ensemble Temperature Scaling for Calibration in Graph Neural Networks (ICLR 2025 · similarity 0.90)
- Understanding and Mitigating Under-Confidence in GNNs from the Final Layer (NeurIPS 2026 · similarity 0.90)
- HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration (NeurIPS 2026 · similarity 0.90)
- Improving GNN Calibration with Discriminative Ability: Insights and Strategies (AAAI 2024 · similarity 0.89)
- Accurate and Scalable Estimation of Epistemic Uncertainty for Graph Neural Networks (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.