Research map / Learning theory and algorithms / Matrix and tensor methods
Computing Higher Order Derivatives of Matrix and Tensor Expressions
Soeren Laue, Matthias Mitterreiter, Joachim Giesen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compute second order derivatives directly on tensor expressions (Ricci calculus) instead of row-by-row Hessian via gradient-times-basis-vector | automatic differentiation framework · Hessian on CPU | running time | Improved |
| same tensor-calculus approach on GPU | automatic differentiation framework · Hessian on GPU | running time | Improved |
| use tensor calculus for first order derivatives vs existing frameworks | automatic differentiation framework · function value and gradient evaluation | running time | No clear effect |
| build expression graph with tensor calculus instead of TensorFlow/Theano | automatic differentiation framework · second order derivative graph creation | graph construction time: a few seconds -> roughly 20 milliseconds | Improved |
Closest papers by meaning
- A Simple and Efficient Tensor Calculus (AAAI 2020 · similarity 0.88)
- Reverse-mode differentiation in arbitrary tensor network format: with application to supervised learning (JMLR 2022 · similarity 0.86)
- Advancing the Lower Bounds: an Accelerated, Stochastic, Second-order Method with Optimal Adaptation to Inexactness (ICLR 2024 · similarity 0.85)
- Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators (NeurIPS 2024 · similarity 0.85)
- Collapsing Taylor Mode Automatic Differentiation (NeurIPS 2025 · similarity 0.84)
- Differentiation Through Black-Box Quadratic Programming Solvers (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
