Research map / Learning theory and algorithms / Sample complexity
On the complexity of nonsmooth automatic differentiation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use backward-mode AD with conservative gradients | Path-differentiable programs · Path-differentiable programs | backward-mode computational overhead: independent of the dimension p | Helped |
| Apply backward-mode AD with conservative gradients to ReLU programs | ReLU programs · ReLU programs | backward-mode computational overhead: 5 | Helped |
| Use forward-mode AD with conservative gradients | Path-differentiable programs · Path-differentiable programs | forward-mode computational overhead: ω_f | Mixed |
| Compute p directional derivatives for a ReLU program | ReLU programs · ReLU programs with p inputs | overhead ratio for computing p directional derivatives: p^{ω - 2 + o(1)} | Hurt |
| Find two distinct Clarke subgradients at a given input | ReLU programs · ReLU programs | complexity of finding two distinct Clarke subgradients: NP-hard | Hurt |
| Find two conservative-gradient elements at a given input instead of enumerating Clarke subgradients | ReLU programs · ReLU programs | complexity of finding two gradient elements: polynomial time | Helped |
| Decide differentiability at a given input | ReLU programs · ReLU programs | complexity of deciding differentiability: NP-hard | Hurt |
Closest papers by meaning
- Directional Smoothness and Gradient Methods: Convergence and Adaptivity (NeurIPS 2024 · similarity 0.87)
- Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates (ICML 2024 · similarity 0.87)
- Automatic Differentiation of Optimization Algorithms with Time-Varying Updates (ICML 2025 · similarity 0.86)
- Gradient Span Algorithms Make Predictable Progress in High Dimension (NeurIPS 2026 · similarity 0.86)
- LPGD: A General Framework for Backpropagation through Embedded Optimization Layers (ICML 2024 · similarity 0.86)
- General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.