Research map / Learning theory and algorithms / Matrix and tensor methods
A Simple and Efficient Tensor Calculus
Sören Laue, Matthias Mitterreiter, Joachim Giesen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-country mode (multiply partial derivatives in increasing tensor order) to reverse mode (ablation) | Einstein-notation tensor calculus · Hessian of logistic regression (CPU) | running time: about 30% faster | Improved |
| add cross-country mode on GPU (ablation) | Einstein-notation tensor calculus · Hessian of logistic regression (GPU) | running time | No clear effect |
| add compression of higher order derivatives (Hessian) (ablation) | Einstein-notation tensor calculus · Matrix factorization Newton system n=1000, k=10 | solve time: 1 sec -> 10 μsec | Improved |
| add compression of Hessian to third order tensor (ablation) | Einstein-notation tensor calculus · small neural net Hessian | running time | Improved |
| use Einstein-notation reverse mode instead of TensorFlow, PyTorch, autograd, JAX for Hessians | Hessian computation · logistic regression, matrix factorization, neural net | running time | Improved |
| enable XLA for TensorFlow on CPU | TensorFlow Hessian · Hessian computation CPU | running time | Hurt |
| enable XLA for TensorFlow on GPU | TensorFlow Hessian · Hessian computation GPU | running time | No clear effect |
Closest papers by meaning
- Computing Higher Order Derivatives of Matrix and Tensor Expressions (NeurIPS 2018 · similarity 0.88)
- It's All Just Vectorization: einx, a Universal Notation for Tensor Operations (ICLR 2026 · similarity 0.88)
- Einsum Benchmark: Enabling the Development of Next-Generation Tensor Execution Engines (NeurIPS 2024 · similarity 0.83)
- The Price of Freedom: Exploring Expressivity and Runtime Tradeoffs in Equivariant Tensor Products (ICML 2025 · similarity 0.82)
- Advancing the Lower Bounds: an Accelerated, Stochastic, Second-order Method with Optimal Adaptation to Inexactness (ICLR 2024 · similarity 0.82)
- Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
