Research map / Graph learning / GNN architectures
GNNX-BENCH: Unravelling the Utility of Perturbation-based GNN Explainers through In-depth Benchmarking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks seven factual and four counterfactual perturbation-based GNN explainers on sufficiency, size, stability (topological noise, explainer seeds, GNN architectures, features, adversarial attacks), reproducibility, and molecular feasibility.
Problem
Existing GNN explainers are evaluated narrowly; stability to noise, seeds and architectures, counterfactual comparisons, and feasibility of counterfactual recourses have not been systematically studied.
Approach
A systematic benchmark of perturbation-based factual and counterfactual GNN explainers across multiple evaluation axes (sufficiency, size, Jaccard stability under noise/seeds/architectures, reproducibility, and feasibility of counterfactuals on molecular graphs), with a released codebase.
Key results
- RCExplainer is the best-performing counterfactual explainer across most datasets and metrics, and was most stable across explainer seeds and noise (e.g., Jaccard ~0.9+ across seeds on Mutagenicity, Proteins, Mutag).
- GNNExplainer shows the lowest factual stability across seeds (Jaccard around 0.5 or lower, e.g., 0.18-0.80), and transductive methods are less stable than inductive ones.
- Counterfactual explanations on molecular datasets deviate significantly from expected connected-graph counts for RCExplainer (Mutagenicity 70 observed vs 233.05 expected, p<0.00001) and CF^2 (0 vs 206.65).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RCExplainer instead of CF² (α=0) for counterfactuals | Graph classification explainers · MUTAG, Mutagenicity, AIDS, Proteins, IMDB-B, ogbg-molhiv | sufficiency and size: MUTAG sufficiency 0.90 ± 0.12 -> 0.4 ± 0.12; Mutagenicity 0.50 ± 0.05 -> 0.4 ± 0.06; AIDS 0.98 ± 0.02 -> 0.91 ± 0.04; Pr | Helped |
| Use CF-GNNExplainer instead of CF² (α=0) | Node classification counterfactual explanations · Tree-Cycles, Tree-Grid, BA-Shapes | accuracy: 100.0 ± 0.0 vs 74.71 ± 18.70; 92.70 ± 4.99 vs 14.29 ± 0.0; 91.5 ± 4.36 vs 68.89 ± 12.28 | Helped |
| Use CLEAR rather than RCExplainer | Graph classification counterfactual explanations · AIDS | sufficiency: 0.91 ± 0.04 -> 0.84 ± 0.03 | Helped |
| Use CLEAR rather than RCExplainer | Graph classification counterfactual explanations · AIDS | size: 1.0 ± 0.0 -> 164.9 ± 47.9 | Hurt |
| Evaluate factual explainers across explainer instances with different seeds | Factual explainers · Mutagenicity, Proteins, Mutag, IMDB-B, AIDS, NCI1 | stability (Jaccard similarity) | Mixed |
| Add topological noise to the input graph | Factual explainers · Mutagenicity, AIDS, Proteins, IMDB-B | sufficiency | Hurt |
| Evaluate counterfactual explanations under topological noise | RCExplainer, CF², and CLEAR · Graph classification datasets | stability (Jaccard similarity) | Mixed |
| Generate counterfactual recourses without enforcing molecular connectivity constraints | RCExplainer and CF² · Mutagenicity, Mutag, AIDS | connected-molecule feasibility: Mutagenicity p-value < 0.00001 for both; AIDS p-value < 0.00001 and 0.0001 | Hurt |
Surprising findings
- CF^2 with alpha=0 (purely counterfactual) underperformed CF-GNNExplainer on node classification, contrary to expectation.
- Retraining GNNs on residual graphs (explanations removed) often recovers original predictions, indicating explainers capture only model-specific signals rather than all data signals.
Limitations
- SubgraphX was run only on test graphs due to scalability, which may bias results.
- CLEAR runs out of memory on larger datasets such as Mutagenicity and PROTEINS, so comparisons are incomplete.
- Feasibility analysis is limited to molecular datasets using connectivity as the constraint.
Experimental setup
Datasets: MUTAG Mutagenicity AIDS PROTEINS IMDB-B NCI1 Graph-SST2 DD REDDIT-B ogbg-molhiv Tree-Cycles Tree-Grid
Benchmarks: MUTAG Mutagenicity AIDS PROTEINS IMDB-B NCI1 ogbg-molhiv Tree-Cycles Tree-Grid BA-Shapes
Metrics: sufficiency necessity explanation size Jaccard stability reproducibility accuracy of ground-truth edges p-value feasibility
Baselines: GNNExplainer PGExplainer TAGExplainer GEM SubgraphX CF-GNNExplainer CF^2 RCExplainer CLEAR
Builds on: GNNExplainer PGExplainer SubgraphX CF-GNNExplainer CF^2 RCExplainer CLEAR
Closest papers by meaning
- Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks (ICLR 2024 · similarity 0.90)
- Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspective (ICML 2025 · similarity 0.90)
- Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNs (ICLR 2025 · similarity 0.89)
- Stratified GNN Explanations through Sufficient Expansion (AAAI 2024 · similarity 0.89)
- Redundancy Undermines the Trustworthiness of Self-Interpretable GNNs (ICML 2025 · similarity 0.89)
- Self-Interpretable Graph Learning with Sufficient and Necessary Explanations (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.