Research map / Optimization / Optimizers
Surfing: Iterative Optimization Over Incrementally Trained Deep Networks
Ganlin Song, Zhou Fan, John Lafferty
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize sequentially across training checkpoints instead of only optimizing the final network | VAE · Fashion-MNIST generated recovery, input dimension 5 | solutions within 0.01 of true latent (%): 99.3 -> 99.3 | No clear effect |
| Optimize sequentially across training checkpoints instead of only optimizing the final network | DCGAN · Fashion-MNIST generated recovery, input dimension 5 | solutions within 0.01 of true latent (%): 50.3 -> 91.3 | Improved |
| Optimize sequentially across training checkpoints instead of only optimizing the final network | WGAN · Fashion-MNIST generated recovery, input dimension 5 | solutions within 0.01 of true latent (%): 48.3 -> 86.7 | Improved |
| Optimize sequentially across training checkpoints instead of only optimizing the final network | WGAN-GP · Fashion-MNIST generated recovery, input dimension 5 | solutions within 0.01 of true latent (%): 46.7 -> 92.0 | Improved |
| Use random initialization as the starting objective, then track the optimum while the network trains | Surfing · Fashion-MNIST generated recovery | solutions within 0.01 of true latent (%) | Mixed |
Closest papers by meaning
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.86)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.86)
- Exact natural gradient in deep linear networks and its application to the nonlinear case (NeurIPS 2018 · similarity 0.86)
- On the Convergence Direction of Gradient Descent (ICLR 2026 · similarity 0.86)
- Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization (ICLR 2020 · similarity 0.86)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
