Research map / Graph learning / GNN architectures
SimCalib: Graph Neural Network Calibration Based on Similarity between Nodes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SimCalib, a post-hoc GNN calibration method that uses Mahalanobis feature similarity to class prototypes and nodewise homophily/relative-degree representation-movement similarity, with a theoretical ECE argument.
Problem
Calibrating confidence of graph neural networks for node classification, where existing GNN calibrators (CaGCN, GATS) lack theoretical grounding and ignore nodewise similarity structure.
Approach
Calibrate frozen GNN logits with nodewise temperatures derived from (i) Mahalanobis distances between node intermediate features and class prototypes propagated over the graph and (ii) an attention weighting based on estimated homophily (logit dot products) and relative degree, combined in a convex mixture; a Gaussian-model analysis argues that jointly estimating correlated nodes lowers ECE.
Key results
- SimCalib achieves best ECE on 14 of 16 backbone-dataset settings, e.g. GCN on Cora 3.32% vs GATS 3.64%.
- Average ECE reduced by 10.4% relative to GATS, with Wilcoxon signed test p=7.63e-4.
- On CoraFull, SimCalib ECE 3.22% (GCN) vs uncalibrated 6.50%, and calibration gains grow as feature similarity decreases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SimCalib instead of the strongest reported baseline | GCN calibration · Cora, Pubmed, Computers, Photo, CS, Physics, CoraFull | ECE (%): Cora 3.64±1.34 -> 3.32±0.99; Pubmed 0.98±0.30 -> 0.93±0.32; Computers 2.58±0.70 -> 1.37±0.33; Photo 1.51±0.52 -> 1.36±0. | Helped |
| Use SimCalib instead of GATS | GAT calibration · Cora, Pubmed, Computers, Photo, Physics, CoraFull | ECE (%): Cora 3.18±0.90 -> 2.90±0.87; Pubmed 1.03±0.32 -> 0.95±0.35; Computers 1.39±0.39 -> 1.08±0.33; Photo 1.48±0.61 -> 1.29±0. | Helped |
| Use SimCalib instead of GATS | GAT calibration · Citeseer | ECE (%): 3.86±1.56 -> 3.95±1.30 | Hurt |
| Use SimCalib instead of GATS | GAT calibration · CS | ECE (%): 0.81±0.30 -> 0.83±0.32 | Hurt |
| Replace class-prototype similarity input with pretrained logits (ablation) | SimCalib · Cora, GCN | ECE (%): 3.32±0.99 -> 3.58±0.97 | Hurt |
| Remove the feature-similarity branch, using representation movement only (ablation) | SimCalib · Cora, GCN | ECE (%): 3.32±0.99 -> 3.86±1.78 | Hurt |
| Remove the relative-degree term (ablation) | SimCalib · Cora, GCN | ECE (%): 3.32±0.99 -> 3.30±1.76 | No clear effect |
| Remove the homophily term (ablation) | SimCalib · Cora, GCN | ECE (%): 3.32±0.99 -> 3.87 ± 1.77 | Hurt |
| Remove homophily and relative-degree terms (ablation) | SimCalib · Cora, GCN | ECE (%): 3.32±0.99 -> 4.16 ±1.79 | Hurt |
| Increase calibration label rate from 1% | SimCalib on GAT · GNN calibration | ECE | Helped |
Surprising findings
- Calibrator improvements are larger for samples with lower feature similarity to class prototypes, linking nodewise similarity to calibration gains.
- The paper reports a link between over-smoothing dynamics and GNN calibration.
Limitations
- Theoretical result relies on a simplified Gaussian mixture model with linear node relationship assumption.
- Slightly worse than GATS on GAT for Citeseer and CS; underperforms at extremely low label rates due to more parameters.
- Homophily is approximated by logit dot products since true labels are unavailable.
Experimental setup
Datasets: Cora Citeseer Pubmed CoraFull Amazon Computers Amazon Photo Coauthor CS Coauthor Physics
Benchmarks: Cora Citeseer Pubmed CoraFull Computers Photo CS Physics
Metrics: ECE ACE
Baselines: UnCal TS VS ETS CaGCN GATS
Builds on: GATS CaGCN temperature scaling Yan et al. two sides of the same coin (representation movement) Lee et al. 2018 Mahalanobis OOD detection
Closest papers by meaning
- Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy (AAAI 2025 · similarity 0.91)
- Understanding and Mitigating Under-Confidence in GNNs from the Final Layer (NeurIPS 2026 · similarity 0.90)
- Improving GNN Calibration with Discriminative Ability: Insights and Strategies (AAAI 2024 · similarity 0.89)
- MaxEnt Loss: Calibrating Graph Neural Networks under Out-of-Distribution Shift (Student Abstract) (AAAI 2024 · similarity 0.88)
- HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration (NeurIPS 2026 · similarity 0.88)
- GETS: Ensemble Temperature Scaling for Calibration in Graph Neural Networks (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.