AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian

Jack Parker-Holder, Luke Metz, Cinjon Resnick, Hengyuan Hu, Adam Lerer, Alistair Letcher, Alexander Peysakhovich, Aldo Pacchiano, Jakob Foerster

NeurIPS 2020 · Optimization › Implicit bias and generalization · approach: SGD noise and flat minima · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use Hessian-eigenvector ridge exploration instead of ERMColored MNIST · Colored MNIST testaccuracy: 17.8 \pm 1.33 -> 58.4 \pm 2.41Improved
Use Hessian-eigenvector ridge exploration instead of ERMColored MNIST · Colored MNIST trainaccuracy: 87.4 \pm 1.70 -> 65.5 \pm 1.68Hurt
Use Ridge Rider rather than IRMColored MNIST · Colored MNISTaccuracy: 69.7 \pm .710 vs 65.5 \pm 1.68; 65.7 \pm 1.42 vs 58.4 \pm 2.41Mixed
Update the eigenvector to follow the changing ridge (ablation)Approximate Ridge Rider · MNISTtest accuracy: approximately 98% vs. beyond 90%Improved
Follow random directions, even when forced to be ascent directions (ablation)MNIST · MNISTaccuracy: does not exceed 30%Hurt
Follow Hessian eigenvectors rather than random directionsRidge Rider · Binary-tree RLpercentage of positive solutions found: over 90% vs. at most 50%Improved
Keep the eigenvector fixed rather than updating it along the ridge (ablation)Ridge Rider · Binary-tree RL, depth 12percentage of solutions foundHurt
Use a fixed step size while continuing to update the ridge (ablation)Ridge Rider · Binary-tree RL, depth 12percentage of solutions foundHurt
Use a fixed step size and fixed ridge (ablation)Ridge Rider · Binary-tree RL, depth 12percentage of solutions foundHurt
Use random unit vectors instead of eigenvectors (ablation)Ridge Rider · Binary-tree RLpercentage of positive solutions foundHurt
Use fixed eigenvectors rather than recomputing them along the path (ablation)Ridge Rider · Binary-tree RL, depth 12percentage of solutions foundNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy