AI Paper Atlas

Research map / Optimization / Optimizers

Approximate Fisher Information Matrix to Characterize the Training of Deep Neural Networks

Zhibin Liao, Tom W. Drummond, Ian D. Reid, Gustavo Carneiro

TPAMI 2018 · Optimization › Optimizers · approach: Memory-efficient LLM optimizers · 33 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase batch size over training (s32-to-512) instead of decreasing (s512-to-32) (ablation)ResNet dynamic sampling · CIFAR-10test error (%): 5.56 -> 5.07Improved
dynamic sampling instead of best fixed batch sizeResNet s32 · CIFAR-10test error (%): 4.78 -> 4.90No clear effect
dynamic sampling instead of best fixed batch sizeResNet s32 · CIFAR-10training time (h): 7.7 -> 7.0Improved
multi-step dynamic sampling (-MS) instead of whole-training dynamic sampling (ablation)ResNet · CIFAR-10test error (%): 4.90 -> 4.76Improved
multi-step dynamic sampling (-MS) instead of whole-training dynamic sampling (ablation)DenseNet · CIFAR-10test error (%): 5.03 -> 4.63Improved
multi-step dynamic sampling (-MS) instead of whole-training dynamic sampling (ablation)DenseNet · CIFAR-100test error (%): 23.68 -> 23.92Hurt
multi-step dynamic sampling (-MS) instead of whole-training dynamic sampling (ablation)ResNet · MNISTtest error (%): 0.39 -> 0.34Improved
multi-step dynamic sampling (-MS) instead of fixed s32DenseNet · CIFAR-10test error (%): 4.96 -> 4.63Improved
dynamic sampling instead of fixed batchDenseNet s128 · SVHNtest error (%): 1.93 -> 2.00No clear effect
dynamic sampling instead of fixed batchDenseNet · MNISTtraining time (h): 0.26 -> 0.06Improved
dynamic sampling instead of fixed batchDenseNet · CIFAR-10training time (h): 16.1 -> 14.1Improved
use small learning rate 0.05 instead of 0.4 with batch size 8 (ablation)ResNet s8 · CIFAR-10test error (%): 17.23 -> 5.09Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy