Research map / Optimization / Optimizers
Tensor Normal Training for Deep Learning Models
Yi Ren, Donald Goldfarb
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TNT instead of SGD-m | ResNet32 · CIFAR-10 validation | validation accuracy: 93.06\% -> 93.08\% | No clear effect |
| Use TNT instead of KFAC | VGG16 · CIFAR-100 validation | validation accuracy: 74.33\% -> 73.33\% | Hurt |
| Use TNT rather than SGD-m | ResNet32 · CIFAR-10 | process time: roughly twice as fast | Improved |
| Use TNT rather than SGD-m | VGG16 · CIFAR-100 | process time: equally as fast | No clear effect |
| Use TNT instead of first-order methods | autoencoder models · MNIST and FACES | training loss optimization | Improved |
| Use TNT instead of KFAC or Shampoo | autoencoder models · MNIST and FACES | training loss optimization | No clear effect |
| Use TNT rather than the empirical Fisher variant (ablation) | TNT · CNN experiments | optimization and generalization performance | Improved |
Closest papers by meaning
- On Tensor Train Rank Minimization : Statistical Efficiency and Scalable Algorithm (NeurIPS 2017 · similarity 0.85)
- Factor Augmented Tensor-on-Tensor Neural Networks (AAAI 2025 · similarity 0.84)
- An Improved Empirical Fisher Approximation for Natural Gradient Descent (NeurIPS 2024 · similarity 0.84)
- A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks (NeurIPS 2024 · similarity 0.84)
- Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent (ICML 2025 · similarity 0.84)
- Geometry-aware training of factorized layers in tensor Tucker format (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
