Research map / Graph learning / GNN architectures

Improving GNN Calibration with Discriminative Ability: Insights and Strategies

AAAI 2024 · Graph learning › GNN architectures · approach: Robustness, privacy, and fairness of GNNs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Argues that GNN post-hoc calibration should also preserve or improve discriminative ability (AUROC) between correct and incorrect predictions, and proposes DC(GNN), an MLP temperature-scaling calibrator fed with discriminative signals including a novel self-loop same-class-neighbor ratio.

Problem

Existing GNN calibration methods reduce calibration error (ECE) but often fail to improve, and sometimes degrade, the ability to discriminate correct from incorrect predictions (AUROC). GNNs are known to be underconfident.

Approach

Diagnoses three causes of calibration failure (over-calibration from regularization biases, insufficient discriminative information in inputs to post-hoc calibrators, and distribution shift between train and test outputs) and builds a node-wise temperature-scaling MLP that takes discriminative signals (confidence, same-class-neighbor ratio, uncertainty mass, and a novel self-loop same-class-neighb

Key results

• DC(GNN) achieves the best ECE and AUROC on nearly all 8 datasets with GCN backbone; e.g., on CS it reaches ECE 1.02% and AUROC 91.22% versus GATS ECE 1.48% and AUROC 88.66%.
• Several existing methods lower ECE while reducing AUROC; e.g., CRL on Pubmed and CaGCN on Cora and Citeseer degrade discriminative ability relative to uncalibrated GCN.
• Ablation shows adding the self-loop same-class-neighbor ratio raises AUROC on Pubmed from 81.62% to 82.47% and on CS from 88.70% to 90.64%.

Measured effects reported in the paper

Surprising findings

• Regularization-based calibration (GCL, CRL) fails to improve ECE significantly or AUROC and sometimes underperforms uncalibrated GCN on Cora and Citeseer.
• Popular calibration methods such as CaGCN and GATS, which use NLL-based training, do not reliably improve AUROC despite being expected to.

Limitations

• Evaluated only on semi-supervised node classification with GCN and GAT backbones.
• Calibration MLP uses only four hand-picked signals; the paper suggests but does not test broader signal sets.
• Reported gains on some datasets are small and DC(GNN)-gt (ground-truth homophily) is outperformed by DC(GNN) on some datasets.

Experimental setup

Datasets: Cora Citeseer Pubmed Computers Photo CS Physics CoraFull

Benchmarks: Cora Citeseer Pubmed Computers Photo CS Physics CoraFull

Metrics: ECE AUROC accuracy

Baselines: Uncal GCL CRL TS VS ETS CaGCN RBS GATS

Builds on: temperature scaling GATS CaGCN RBS

Closest papers by meaning

• Understanding and Mitigating Under-Confidence in GNNs from the Final Layer (NeurIPS 2026 · similarity 0.92)
• MaxEnt Loss: Calibrating Graph Neural Networks under Out-of-Distribution Shift (Student Abstract) (AAAI 2024 · similarity 0.89)
• Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy (AAAI 2025 · similarity 0.89)
• SimCalib: Graph Neural Network Calibration Based on Similarity between Nodes (AAAI 2024 · similarity 0.89)
• Online GNN Evaluation Under Test-time Graph Distribution Shifts (ICLR 2024 · similarity 0.88)
• Moderate Message Passing Improves Calibration: A Universal Way to Mitigate Confidence Bias in Graph Neural Networks (AAAI 2024 · similarity 0.88)