Research map / Graph learning / GNN architectures
Effects of Dropout on Performance in Long-range Graph Learning Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretically and empirically shows that DropEdge-style and Dropout-style methods reduce sensitivity between distant nodes (worsening over-squashing), and proposes DropSens, a sensitivity-aware DropEdge variant that outperforms graph rewiring on long-range node and graph classification benchmarks.
Problem
Random edge-dropping and dropout methods (DropEdge, DropNode, DropAgg, DropGNN, Dropout, DropMessage) were designed to fight over-smoothing, but their effect on over-squashing and long-range interactions in MPNNs is unexplored.
Approach
Analyze the expected sensitivity of node representations under random edge masks in linear GCNs, showing DropEdge-variants decay sensitivity to distant nodes; then design DropSens, which sets a per-node edge-drop probability from in-degree so a fixed fraction of information over cross-edges is preserved.
Key results
- DropSens achieves 84.98 on Cora, 84.30 on PubMed, and 53.01 on Chameleon with GCN, outperforming the listed rewiring methods on node classification.
- On graph classification, DropSens reaches 36.01 on Enzymes, 77.44 on Reddit and 61.68 on Collab, the best among compared rewiring methods on those datasets.
- Dropout-variants improve homophilic node classification in 40/54 cases but the best dropout configuration underperforms NoDrop in 45/54 heterophilic cases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use sensitivity-aware edge dropping (DropSens) instead of graph rewiring baselines | GCN · Node classification: Cora, CiteSeer, PubMed, Chameleon, Squirrel, Actor | accuracy | Mixed |
| Use sensitivity-aware edge dropping (DropSens) instead of graph rewiring baselines | GCN · Graph classification: Mutag, Proteins, Enzymes, Reddit, IMDb, Collab | accuracy | Mixed |
| Apply DropEdge rather than NoDrop | GCN · Homophilic node-classification datasets | test accuracy: +0.419 on Cora; +0.686 on CiteSeer; +0.385 on PubMed | Helped |
| Apply dropout variants rather than NoDrop | GCN and GIN · Heterophilic node-classification datasets | test accuracy: -10.640 on Chameleon; -13.970 on Squirrel; -6.674 on TwitchDE (GCN DropAgg) | Hurt |
| Apply dropout variants rather than NoDrop | GCN and GIN · Graph-classification datasets | test accuracy: -16.460 on Reddit; +3.580 on IMDb (GCN DropAgg) | Mixed |
| Use DropSens rather than DropEdge | GCN · Long-range real-world tasks | error rate | Helped |
| Increase edge-dropping probability | L-layer linear GCN with DropEdge, DropAgg, or DropGNN · Nodes exactly L hops apart | sensitivity | Hurt |
Surprising findings
- DropSens does not perform competitively with GIN, which is unsurprising since it was designed around GCN message passing.
- Dropout methods often have non-positive effects on graph-classification tasks (67/108 cases), suggesting over-squashing may matter less at graph level.
Limitations
- Theoretical analysis relies on linear GCN and simplifying assumptions that may not generalize to ResGCN, GAT or graph transformers.
- DropSens requires an architecture-specific alteration to the edge-dropping strategy and is not general.
- Hyperparameter search over 20 runs and t-tests on only 50 samples per configuration.
Experimental setup
Datasets: SyntheticZINC Cora CiteSeer PubMed Chameleon Squirrel TwitchDE Mutag Proteins Enzymes Reddit IMDb
Benchmarks: SyntheticZINC Cora CiteSeer PubMed Chameleon Squirrel TwitchDE Mutag Proteins Enzymes Reddit IMDb
Metrics: test accuracy MAE p-value (t-test) Hedges' g
Baselines: DropEdge DropNode DropAgg DropGNN Dropout DropMessage FA DIGL SDRF FoSR GTR
Builds on: DropEdge DropMessage DropGNN GCN Jacobian-based over-squashing analysis
Closest papers by meaning
- Beyond Random Masking: When Dropout meets Graph Convolutional Networks (ICLR 2025 · similarity 0.91)
- Aggregation Buffer: Revisiting DropEdge with a New Parameter Block (ICML 2025 · similarity 0.87)
- On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning (NeurIPS 2025 · similarity 0.87)
- Spectral Graph Pruning Against Over-Squashing and Over-Smoothing (NeurIPS 2024 · similarity 0.87)
- The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisited (ICLR 2025 · similarity 0.86)
- Get a GRIP, this will be a long TRIP: A Quantifiable Long-Range Framework for Verifying Over-squashing (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.