Research map / Adversarial robustness and security / Attacks on text and graphs
If You Want to Be Robust, Be Wary of Initialization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives upper bounds on adversarial risk of GCN, GIN and general DNNs linking robustness to weight initialization norms and number of training epochs, and validates trends empirically on Cora, CiteSeer, ACM and MNIST.
Problem
Understanding how weight initialization and training duration affect the adversarial robustness of graph neural networks (and neural networks in general), an under-explored dimension compared with pre-processing and message-passing defenses.
Approach
Theoretical upper bounds on adversarial risk under gradient descent with L-smooth losses show the bound grows with initial weight norms and with training epochs (via the 2^t factor), so initialization choice and early stopping trade off clean accuracy against robustness; the bound is extended to GIN, general DNNs, and strongly convex losses.
Key results
- Attacked-accuracy gaps between initialization schemes reach up to about 50% per the abstract; for certain budgets (30% and 40%) the success-rate gap under Gaussian variance changes ranges between 5% and 15% on Cora/CiteSeer.
- On DNNs (MNIST), the accuracy gap between best and worst initialization at epsilon=0.1 is around 60%.
- Attacked accuracy rises then falls with training epochs, showing a clean-vs-robust trade-off with an inflection point where early stopping is advisable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase Gaussian initialization variance | GCN · Cora and CiteSeer, PGD and Mettack | Success Rate: between $5\%$ and $15\%$ | Hurt |
| Increase initialization scaling parameter β | GCN with Uniform or Orthogonal initialization · Cora and CiteSeer, PGD and Mettack | Success Rate | Hurt |
| Train for more epochs | GCN · Cora and CiteSeer under adversarial attacks | Attacked Accuracy | Mixed |
| Choose different initialization schemes | GIN on Cora and DNN on MNIST · PGD at ε=0.1 | Attacked Accuracy: around $60\%$ | Mixed |
| Choose different initialization schemes | GIN on Cora and DNN on MNIST · PGD, clean evaluation (ε=0) | Clean accuracy | No clear effect |
Surprising findings
- Initializing weights to zero minimizes the bound but tends to yield poor learning, so the bound alone does not select the initialization.
- Robustness gains from initialization are reported as almost free, with clean accuracy largely unchanged across initializations.
Limitations
- The paper does not propose a new initialization scheme or defense; it offers analysis only.
- Theory relies on strong assumptions such as L-smoothness, 1-Lipschitz activations, gradient descent and a local optimum W*, and the bounds are loose (e.g., 2^t factor).
- Experiments are limited to small citation networks, MNIST and ACM with a 2-layer GCN and a few attacks.
Experimental setup
Datasets: Cora CiteSeer ACM MNIST
Benchmarks: Cora CiteSeer ACM MNIST
Metrics: attacked accuracy success rate clean accuracy
Baselines: Gaussian initialization Uniform initialization Orthogonal initialization Kaiming initialization Xavier initialization RGCN GCN-Jaccard
Builds on: GCN Mettack Parseval networks GCORN
Closest papers by meaning
- Are Defenses for Graph Neural Networks Robust? (NeurIPS 2022 · similarity 0.92)
- Boosting the Adversarial Robustness of Graph Neural Networks: An OOD Perspective (ICLR 2024 · similarity 0.91)
- Value at Adversarial Risk: A Graph Defense Strategy against Cost-Aware Attacks (AAAI 2024 · similarity 0.91)
- Bounding the Expected Robustness of Graph Neural Networks Subject to Node Feature Attacks (ICLR 2024 · similarity 0.90)
- Adversarial Robust Generalization of Graph Neural Networks (ICML 2025 · similarity 0.90)
- A Simple and Yet Fairly Effective Defense for Graph Neural Networks (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.