Research map / Adversarial robustness and security / Black-box and transfer attacks
A Theoretical View of Linear Backpropagation and its Convergence
Ziang Li, Yiwen Guo, Haodi Liu, Changshui Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LinBP instead of BP for gradients | PGD white-box attack · CIFAR-10 ResNet-50, K=10, eps=8/255 | attack success rate: 66.41% -> 85.83% | Improved |
| use LinBP instead of BP in PGD with 100 steps | PGD white-box attack · CIFAR-10 non-robust models, K=100 | attack success rate: 99.73% -> 100.00% | No clear effect |
| use LinBP instead of BP | PGD attack on robust WRN-28-10 · CIFAR-10 K=10 eps=8/255 | attack success rate: 33.14% -> 33.48% | Improved |
| use LinBP instead of BP | Auto-PGD attack · CIFAR-10 ResNet-50 eps=8/255 | attack success rate: 95.77% -> 97.60% | Improved |
| use LinBP so Auto-PGD halves step size sooner | Auto-PGD on ResNet-50 · CIFAR-10 eps=8/255 | iterations before first step halving: 22.95 vs 25.85 | Improved |
| use LinBP with Swish/SiLU activations | PGD on ResNet-50 (SiLU) · CIFAR-10 K=10 eps=16/255 | attack success rate: 82.65% -> 87.97% | Improved |
| use LinBP instead of BP gradients with l2 gradient normalization in simulated model training | one-layer teacher-student · synthetic training | l1 distance to teacher: similar training loss and approximation | No clear effect |
| use LinBP gradients for training | MLP and LeNet-5 SGD · MNIST | training loss and accuracy: lower loss, higher accuracy early | Improved |
| use LinBP gradients for training | ResNet-50, DenseNet-161, MobileNetV2 · CIFAR-10 | final accuracy: comparable to BP at the end | No clear effect |
| scale learning rate 10x with LinBP | LeNet-5 SGD · MNIST | training convergence: failed to converge with LinBP while BP converged | Hurt |
| use LinBP gradients for training with normalized gradients | DNN training · CIFAR-10/MNIST | training performance: LinBP no better than BP | No clear effect |
| generate adversarial examples with LinBP during adversarial training (second strategy) | adversarial training · CIFAR-10 robustness to BP-generated examples | test robust accuracy: larger generalization gap, degraded robustness | Hurt |
Closest papers by meaning
- Overcoming Rank Collapse in Feedback Alignment (NeurIPS 2026 · similarity 0.83)
- Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation (ICML 2024 · similarity 0.83)
- Inspecting Prediction Confidence for Detecting Black-Box Backdoor Attacks (AAAI 2024 · similarity 0.83)
- Efficient Target Propagation by Deriving Analytical Solution (AAAI 2024 · similarity 0.83)
- One Forward is Enough for Neural Network Training via Likelihood Ratio Method (ICLR 2024 · similarity 0.83)
- Fusing Pruned and Backdoored Models: Optimal Transport-based Data-free Backdoor Mitigation (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
