AI Paper Atlas

Research map / Optimization / Optimizers

SEBOOST - Boosting Stochastic Learning Using Subspace Optimization Techniques

Elad Richardson, Rom Herskovitz, Boris Ginsburg, Michael Zibulevsky

NeurIPS 2016 · Optimization › Optimizers · approach: Differentiating through embedded optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add SEBOOST subspace optimization on top of SGDSGD baseline · MNIST autoencoderreconstruction error / convergenceImproved
add SEBOOST subspace optimization on top of NAGNAG baseline · MNIST autoencoderreconstruction error / convergenceImproved
add SEBOOST subspace optimization on top of AdaGradAdaGrad baseline · MNIST autoencoderreconstruction error / convergenceImproved
add SEBOOST subspace optimization on top of SGD and NAGSGD and NAG baselines · CIFAR-10 ResNet-32final minimum / accuracyImproved
add SEBOOST subspace optimization on top of AdaGradAdaGrad baseline · CIFAR-10 ResNet-32final minimum / accuracyNo clear effect
add SEBOOST on top of SGD with a simple regression networkSGD baseline · Simple regression (20,000 samples, 6->12->8->4->1)convergence speedImproved
apply subspace optimization too frequently (small ell) (ablation)SEBOOST · MNIST autoencoderruntime and final minimumHurt
increase M from small to 50 (larger subspace) (ablation)SEBOOST · MNIST autoencoderfinal minimum vs early descent speedMixed
add anchor points to the subspace (ablation)SEBOOST · MNIST autoencoderreconstruction error / convergenceImproved
add momentum direction to the subspace (ablation)SEBOOST · MNIST autoencoderconvergenceMixed
change baseline learning rate (robustness test) (ablation)SEBOOST boosted variants · MNIST autoencoderconvergence sensitivity to learning rateImproved
apply boosting step once per epoch vs too frequently or too infrequently (ablation)SEBOOST on ResNet-32 · CIFAR-10 ResNet-32final minimumMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy