Research map / Probabilistic and scientific modeling / Variational and generative sampling
Great Minds Think Alike: The Universal Convergence Trend of Input Salience
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies the distribution of stochastically optimized DNNs via input saliency maps, showing that larger models' gradient directions converge to a shared population mean and modeling the convergence with a Saw distribution.
Problem
Understanding the distribution of optimized DNNs induced by stochastic training, specifically how input saliency (input gradient directions) varies across models of different capacities and families.
Approach
Views input-gradient directions of optimized models as points on a hypersphere and hypothesizes that within an architecture, family mean directions align while the spherical variance shrinks with width, so the distribution converges to a shared mean; models this with a semi-parametric Saw distribution and applies it to black-box attacks and deep ensembles.
Key results
- Cosine similarity between input saliency of models increases with width on CIFAR-10/100 and TinyImagenet, with mean direction similarity near 1.0 within each architecture.
- Mean-direction attacks transfer much more successfully than single-model attacks across capacities on CIFAR-10.
- Expected testing loss is highly correlated with the marginal expectation E[t] for both single models and deep ensembles.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase model width | CNN and ResNet families · CIFAR-10 and CIFAR-100 | input-gradient cosine similarity | Helped |
| Increase model capacity | Optimized CNN and ResNet models · CIFAR-10, CIFAR-100, TinyImagenet | within-family input-gradient similarity | Helped |
| Use a larger-capacity model as the black-box attack source | Single-model black-box attack · CIFAR-10 | attack success | Helped |
| Use the estimated mean gradient direction instead of a single-model gradient | Black-box attack · CIFAR-10 | attack success | Helped |
| Initialize model parameters without optimization (ablation) | CNN and ResNet model families · CIFAR-10, CIFAR-100, TinyImagenet | input-saliency cosine similarity trend with capacity | Hurt |
| Use predicted probabilities rather than predicted logits to generate saliency maps (ablation) | CNN and ResNet models · CIFAR-10, CIFAR-100, TinyImagenet | input-saliency cosine similarity trend with capacity | No clear effect |
| Vary batch size among 64, 128, 256, and 512 (ablation) | CNN and ResNet training · CIFAR-10 | input-gradient cosine similarity trend with capacity | No clear effect |
| Vary learning rate among 1e-1, 1e-2, and the default schedule (ablation) | CNN and ResNet training · CIFAR-10 | input-gradient cosine similarity trend with capacity | No clear effect |
| Replace SGD with Adam, AdamW, or SGD with momentum (ablation) | CNN training · CIFAR-10 | input-gradient cosine similarity trend with capacity | No clear effect |
| Increase network depth while holding layer widths fixed (ablation) | CNN · CIFAR-10 | input-gradient cosine similarity | Mixed |
| Fix the initialization while varying training-batch order (ablation) | CNN, k=40 · CIFAR-10 | mean ρind: 0.0753 -> 0.0879; 0.0753 -> 0.0855 | Helped |
| Use a vision transformer architecture (ablation) | ViT · CIFAR-10 | input-gradient cosine similarity trend with capacity | No clear effect |
Surprising findings
- Larger models resemble smaller models more than smaller models resemble themselves, i.e., cross-family similarity exceeds intra-family similarity.
- Randomly initialized models show cosine similarity concentrated near 0, indicating the trend is caused by optimization.
Limitations
- Experiments limited to rather small datasets (CIFAR-10/100, TinyImagenet) due to computational burden.
- Hypotheses are empirically verified and only partially theoretically justified; symmetry of Saw distribution is checked empirically.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImagenet-200
Benchmarks: CIFAR-10 CIFAR-100 TinyImagenet-200
Metrics: cosine similarity of input gradients spherical variance / mean resultant length Wasserstein distance black-box attack rate testing loss
Baselines: single-model black-box attack deep ensembles
Builds on: benign overfitting Saw distribution deep ensembles Nakkiran et al. 2021 deep double descent setup
Closest papers by meaning
- On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective (NeurIPS 2023 · similarity 0.87)
- CAMERAS: Enhanced Resolution and Sanity Preserving Class Activation Mapping for Image Saliency (CVPR 2021 · similarity 0.87)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.86)
- On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling (NeurIPS 2025 · similarity 0.85)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.85)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.