Research map / Learning theory and algorithms / Prediction and decision losses
On the Correctness of Automatic Differentiation for Neural Networks with Machine-Representable Parameters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Give every layer bias parameters | Neural networks using pointwise continuous piecewise-analytic activations · Differentiable parameters | incorrect set size: incorrect set is always empty | Helped |
| Give every layer bias parameters | Neural networks with machine-representable parameters · Non-differentiable parameters | non-differentiable set density: \frac{1}{|\mathboldcommand{M}|} {\sum_{(l,i) \in \mathsf{Idx}}} | \ndf{\sigma_{l,i}} | | Helped |
| Use pre-activation functions without bias parameters | Neural networks · Differentiable parameters | incorrect set density: can be non-empty | Hurt |
| Use pre-activation functions without bias parameters | Neural networks · Non-differentiable and incorrect parameters | combined set density: \frac{1}{|\mathboldcommand{M}|} {\sum_{(l,i) \in \mathsf{Idx}}} \Big| \ndf{\sigma_{l,i}} \cup \big(\bdz{\sigma_{l,i}} \c | Mixed |
| Use consistent extended derivatives for activations | Neural networks with bias parameters · Non-differentiable parameters | generalized derivative correctness: AD computes a Clarke subderivative | Helped |
| Require no activation non-differentiabilities at neurons whose layers lack biases, subject to the stated conditions | General neural networks · Parameters satisfying the sufficient condition | standard derivative correctness: AD(w) = Df(w) != bottom | Helped |
Closest papers by meaning
- What does automatic differentiation compute for neural networks? (ICLR 2024 · similarity 0.93)
- On Correctness of Automatic Differentiation for Non-Differentiable Functions (NeurIPS 2020 · similarity 0.89)
- Floating-Point Networks with Automatic Differentiation Can Represent Almost All Floating-Point Functions and Their Gradients (ICML 2026 · similarity 0.87)
- A mathematical model for automatic differentiation in machine learning (NeurIPS 2020 · similarity 0.85)
- Taming Binarized Neural Networks and Mixed-Integer Programs (AAAI 2024 · similarity 0.85)
- Floating-Point Neural Networks Can Represent Almost All Floating-Point Functions (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.