Research map / Architectures and efficiency / RNNs and normalization
Faster Neural Network Training with Approximate Tensor Operations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply top-k approximate tensor operations | WRN-28-10 · CIFAR-10 | training speedup: 1.33x | Helped |
| Apply top-k approximate tensor operations | CNN · MNIST | accuracy: 99.35% -> 99.25% | Hurt |
| Apply approximate tensor operations | MLP · MNIST | accuracy: 98.22% -> 98.22% | No clear effect |
| Apply approximate tensor operations | WRN-28-10 · CIFAR-10 | accuracy: 96.17% -> 95.89% | Hurt |
| Apply approximate tensor operations | ResNet-50 · ImageNet | accuracy: 75.6% -> 75.63% | No clear effect |
| Use top-k selection instead of CRS or Bernoulli-CRS (ablation) | Approximate tensor operations · MNIST and CIFAR-10 | accuracy | Helped |
| Approximate only the backward pass rather than consistently sampling forward and backward (ablation) | WRN-28-10 · CIFAR-10 | accuracy | Hurt |
| Approximate only the backward pass (ablation) | Small MNIST models · MNIST | accuracy: 10% | No clear effect |
| Approximate only the forward pass and propagate gradients as if computation were exact (ablation) | Deep neural networks · Not specified | accuracy | Hurt |
| Use top-k-weights sampling and communicate only sampled gradients | ResNet-152 · ImageNet, 8 nodes | training speedup: 1.37x | Helped |
Closest papers by meaning
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.88)
- Learning to Optimize Tensor Programs (NeurIPS 2018 · similarity 0.87)
- Sampling weights of deep neural networks (NeurIPS 2023 · similarity 0.86)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.85)
- Geometry-aware training of factorized layers in tensor Tucker format (NeurIPS 2024 · similarity 0.85)
- Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.