Research map / Optimization / Optimizers
On Training Implicit Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace implicit differentiation with unrolling phantom gradient (UPG) | MDEQ · CIFAR-10 | accuracy (%): 93.8 -> 95.0 | Helped |
| replace implicit differentiation with UPG | MDEQ · ImageNet | accuracy (%): 75.3 -> 75.7 | Helped |
| replace implicit differentiation with UPG | DEQ (PostLN) · Wikitext-103 | perplexity: 24.0 -> 25.7 | Hurt |
| replace implicit differentiation with UPG under Jacobian regularization | DEQ (PreLN) · Wikitext-103 | perplexity: 24.5 -> 24.4 | No clear effect |
| replace implicit differentiation with UPG | IGNN · COX2 / PROTEINS / PPI | accuracy / Micro-F1: PPI 97.6 -> 98.2; COX2 84.1 -> 83.9 | No clear effect |
| increase damping factor lambda from 0.5 to 1.0 in UPG (ablation) | IGNN · PPI | Micro-F1 (%): 98.2 -> 96.2 | Hurt |
| add unrolled pretraining stage (ablation) | MDEQ-Tiny implicit differentiation · CIFAR-10 ablation setting | accuracy (%): 82.3 -> 85.0 | Helped |
| add dropout (ablation) | MDEQ-Tiny implicit differentiation · CIFAR-10 ablation setting | accuracy (%): 83.7 -> 85.0 | Helped |
| use Adam instead of SGD (ablation) | MDEQ-Tiny implicit differentiation · CIFAR-10 ablation setting | accuracy (%): 84.5 -> 85.0 | Helped |
| use UPG phantom gradient instead of implicit differentiation (no dropout) (ablation) | MDEQ-Tiny · CIFAR-10 ablation setting | accuracy (%): 85.0 -> 85.8 | Helped |
| use unrolling (UPG) instead of Neumann series (NPG) (ablation) | phantom gradient · CIFAR-10 ablation setting | accuracy (%): 85.6 -> 85.8 | Helped |
| use 9 unrolling steps with dropout (ablation) | UPG · CIFAR-10 ablation setting | accuracy (%): 85.8 -> 86.1 | Helped |
Closest papers by meaning
- Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent (ICML 2025 · similarity 0.85)
- Implicit regularization of deep residual networks towards neural ODEs (ICLR 2024 · similarity 0.84)
- Mask in the Mirror: Implicit Sparsification (ICLR 2025 · similarity 0.84)
- On the Implicit Bias of Adam (ICML 2024 · similarity 0.84)
- Subhomogeneous Deep Equilibrium Models (ICML 2024 · similarity 0.84)
- Continuous Semi-Implicit Models (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.