AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

On the Angular Update and Hyperparameter Tuning of a Scale-Invariant Network

Juseung Yun, Janghyeon Lee, Hyounguk Shon, Eojindl Yi, Seung Hwan Kim, Junmo Kim

ECCV 2022 · Architectures and efficiency › RNNs and normalization · approach: Theory of depth and width · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
raise LR from 0.1 to 3.2 to match tuning factor at N=3125 (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=3125test accuracy (%): 71.62 -> 78.28Improved
lower LR to 0.2 vs 0.1 at N=50000 (matched factor) (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=50000test accuracy (%): 94.86 -> 95.08No clear effect
increase LR to 6.4 beyond tuning factor (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=50000test accuracy (%): 95.13 -> 80.28Hurt
increase WD to 0.0064 beyond tuning factor (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=50000test accuracy (%): 95.13 -> 80.18Hurt
scale LR with batch size (linear scaling) holding N fixed (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=50000test accuracy (%): 94.87 to 95.13No clear effect
scale WD linearly with batch size instead of LR (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=50000test accuracy (%): 95.30 best vs yellow cellsNo clear effect
scale batch size with N at fixed LR and WD (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10test accuracy (%): 51.51 -> 79.25Improved
use tuning factor with SGD (no momentum) (ablation)SGD scale-invariant ResNet-18 · CIFAR-10 B>512test accuracy (%): second-best value not in yellow cellMixed
use cosine LR schedule (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=50000test accuracy (%): 95.08 -> 95.14No clear effect
remove scale-invariant modification (unmodified ResNet-18) (ablation)SGDM ResNet-18 · CIFAR-10 N=50000test accuracy (%): 95.08 -> 95.09No clear effect
tune WD on 80k subset then rescale by tuning factorSGDM EfficientNet-B0 · ImageNettest accuracy (%): best lambda 0.0016 -> predicted 1e-04 vs best 5e-05Improved
reduce batch size from 128 to 64 at N=3125 (fixed LR WD) (ablation)SGDM scale-invariant ResNet-18 · CIFAR-10 N=3125test accuracy (%): 77.57 -> 79.25Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy