Research map / Graph learning / GNN architectures

How Interpretable Are Interpretable Graph Neural Networks?

ICML 2024 · Graph learning › GNN architectures · approach: GNN explainability · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a subgraph multilinear extension (SMT) framework to analyze interpretable GNNs, shows attention-based XGNNs fail to approximate SMT, and introduces GMT (a linearized variant and a random-subgraph-sampling variant with classifier retraining) that improves interpretability and OOD generalization.

Problem

Attention-based interpretable GNNs (XGNNs) extract subgraphs via soft attention, but their representational properties are poorly understood and they may not faithfully approximate the label distribution over sampled subgraphs, degrading interpretability and OOD generalization.

Approach

Model interpretable subgraph learning as the multilinear extension of the subgraph predictivity (SMT); show that feeding the expected soft subgraph into a non-linear GNN breaks this equality, and fix it with either a 1-round linearized message passing variant or Monte Carlo random subgraph sampling plus a retrained classifier on frozen extractor.

Key results

• On Spurious-Motif (GIN, b=0.5), GMT-sam improves interpretation AUROC over GSAT from 74.49 to 85.50 (about 11 points, with the 15% headline claim for the GIN setting).
• With classifier retraining on frozen extractor, GMT-sam (PNA) reaches 71.98% accuracy on SPMotif-0.5 versus 47.45% for GSAT (GIN) originally trained classifier.
• On geometric graphs, GMT-sam achieves ROC AUC 98.55 on ActsTrack and 99.89 on SynMol versus ERM baselines of 97.40 and 99.30.

Measured effects reported in the paper

Surprising findings

• GSAT's counterfactual fidelity estimated via random attention perturbation is 2 to 3 times lower than simulated SMT, indicating poor SMT approximation.
• Linearized GMT-lin (one round of message passing) already improves interpretability over GSAT despite its simplicity.

Limitations

• Computational overhead of random subgraph sampling for large-scale node classification
• Sampling rounds can degrade optimization beyond saturation
• Fine-grained node-feature interpretation not addressed

Experimental setup

Datasets: BA-2Motifs Mutag MNIST-75sp Spurious-Motif Graph-SST2 OGBG-MolHIV ActsTrack Tau3Mu SynMol PLBind Cora Coauthor-Physics

Benchmarks: BA-2Motifs Mutag MNIST-75sp Spurious-Motif Graph-SST2 OGBG-MolHIV ActsTrack Tau3Mu SynMol PLBind

Metrics: AUROC Precision@5 Average Precision accuracy ROC AUC counterfactual fidelity sufficiency fidelity

Baselines: GSAT LRI-Bernoulli DIR IB-subgraph GNNExplainer PGExplainer GraphMask GradGeo

Builds on: GSAT LRI graph information bottleneck multilinear extension

Closest papers by meaning

• From GNNs to Trees: Multi-Granular Interpretability for Graph Neural Networks (ICLR 2025 · similarity 0.90)
• Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks (ICLR 2024 · similarity 0.87)
• Generating In-Distribution Proxy Graphs for Explaining Graph Neural Networks (ICML 2024 · similarity 0.87)
• Graph Segmentation and Contrastive Enhanced Explainer for Graph Neural Networks (AAAI 2025 · similarity 0.87)
• Translating Subgraphs to Nodes Makes Simple GNNs Strong and Efficient for Subgraph Representation Learning (ICML 2024 · similarity 0.87)
• Self-Interpretable Graph Learning with Sufficient and Necessary Explanations (AAAI 2024 · similarity 0.87)