Research map / Learning theory and algorithms / Prediction and decision losses
Theory of neural network sensitivity
Analyze learnability and robustness of networks through linear structure and Lipschitz control.
143 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Prediction and decision losses, keyword cluster “selection · robust · test”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
48 papers in 2023–24 and 27 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 9.0% to 4.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Efficient Neural Codes under Metabolic Constraints (NeurIPS 2016 · 9 citations)
- 2017 · Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice (NeurIPS 2017 · 144 citations)
- 2018 · Look at Boundary: A Boundary-Aware Face Alignment Algorithm (CVPR 2018 · 534 citations)
- 2019 · Controlling Neural Level Sets (NeurIPS 2019 · 62 citations)
- 2020 · Neural Networks Fail to Learn Periodic Functions and How to Fix It (NeurIPS 2020 · 55 citations)
- 2021 · HyDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks (AAAI 2021 · 25 citations)
- 2022 · Locally Sparse Neural Networks for Tabular Biomedical Data (ICML 2022 · 11 citations)
- 2023 · Perfectly Secure Steganography Using Minimum Entropy Coupling (ICLR 2023 · 12 citations)
- 2024 · Circuit Design and Efficient Simulation of Quantum Inner Product and Empirical Studies of Its Effect on Near-Term Hybrid Quantum-Classic Machine Learning (CVPR 2024 · 9 citations)
- 2025 · Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Time (CVPR 2025 · 4 citations)
- 2026 · Patterning: The Dual of Interpretability (ICML 2026 · 0 citations)
Earliest papers here
- Efficient Neural Codes under Metabolic Constraints (NeurIPS 2016 · 9 citations)
- On the Quality of the Initial Basin in Overspecified Neural Networks (ICML 2016 · 7 citations)
- Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice (NeurIPS 2017 · 144 citations)
What has been tried
116 papers here report 1,056 measured changes (a component added, removed or swapped, compared with a baseline): helped 65%, no clear effect 9%, hurt 20%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add quantum state search | t=6, r=1 · running time | 44.86 s -> 4.74 s (9.45x) | Helped | Circuit Design and Efficient Simulation of Quantum Inner Product and Empirical Studies of Its Effect on Near-Term Hybrid Quantum-Classic Machine Learning (CVPR 2024 · 9 citations) |
| Add weight decay to PEPITA | CIFAR-10 · accuracy | 52.45±0.25 -> 53.05±0.23 | Helped | Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization (ICLR 2024 · 7 citations) |
| Train with the BER objective only instead of combined objectives | High SNR · LER | 27 times higher LER | Hurt | Deep Quantum Error Correction (AAAI 2024 · 6 citations) |
| Use H1 instead of H5 split heuristic | Model_2_20 · safe rate | 78.5% -> 78.91% | Helped | Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees (AAAI 2024 · 6 citations) |
| Add Gaussian white noise to the target | Feynman LP subset · solution rate | Mixed | GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations (AAAI 2024 · 3 citations) | |
| increase network width from 5 to 10 neurons | Σ2 · JSR approximation precision | 9.0251 -> 8.6969 | Helped | Stability Analysis of Switched Linear Systems with Neural Lyapunov Functions (AAAI 2024 · 3 citations) |
| Increase CNN7 width from 1× to 2× | CIFAR-10 ε=2/255 · certified accuracy | Helped | Understanding Certified Training with Interval Bound Propagation (ICLR 2024 · 2 citations) | |
| Add Jacobian homeostatic loss | CIFAR-10 validation · Top-1 accuracy (%) | 60.4 ± 0.4 -> 84.3 ± 0.1 | Helped | Improving equilibrium propagation without weight symmetry through Jacobian homeostasis (ICLR 2024 · 2 citations) |
| use a deformed metric with selected hyperparameters rather than the standard metric | Radar and HDM05 · accuracy | Helped | RMLR: Extending Multinomial Logistic Regression into General Geometries (NeurIPS 2024 · 2 citations) | |
| Increase the number of retained pathways/features | Synthetic multivariable case · test-set MSE | Mixed | Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · 1 citation) | |
| Apply the learned adaptive-temperature function only at inference | CLRS-Text · accuracy | Hurt | Softmax is not Enough (for Sharp Size Generalisation) (ICML 2025 · 1 citation) | |
| Add data augmentation to pcDEQ-1-MC-ReLU6 | CIFAR-10 test · accuracy | 78.2 -> 89.2 | Helped | Positive Concave Deep Equilibrium Models (ICML 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 18 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MNIST · 4 of 18 papers
- CIFAR-10 · 2 of 18 papers
- Tiny ImageNet · 2 of 18 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 75 papers; from OpenAlex and the papers' affiliations).
- University of Oxford · 5 papers
- Swiss Federal Institute of Technology · 4 papers
- ETHZ - ETH Zurich · 3 papers
- Google · 3 papers
- Meta · 2 papers
- Carnegie Mellon University · 2 papers
- EPFL · 2 papers
- Korea University · 2 papers
- Harvard University · 2 papers
- University of California, Berkeley · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Active learning and subset selection (199 papers)
- Adaptive testing and experiments (201 papers)
- Distributionally robust optimization (184 papers)
- Data attribution and recourse (160 papers)
- Determinantal point process inference (122 papers)
Elsewhere in the topic
- Ranking models from comparisons (127 papers)
- Conformal prediction sets (116 papers)
- Provable post-hoc explanations (92 papers)
- Tree ensemble optimization (77 papers)
- Optimal decision tree learning (75 papers)
- Consistency of surrogate losses (72 papers)
- Losses for decision-making (68 papers)
- Shapley value attribution (63 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).