Research map / Optimization / Optimizers
Improving Gradient Flow with Unrolled Highway Expectation Maximization
Chonghyuk Song, Eunseok Kim, Inwook Shim
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the EM M-step with a Newton-Raphson GEM update that adds a scaled skip connection | EM attention with ResNet-101 · PASCAL VOC val (SS, 256 channels) | mIoU: 79.73 -> 80.93 | Improved |
| Replace the EM M-step with a Newton-Raphson GEM update that adds a scaled skip connection | EM attention with ResNet-101 · PASCAL VOC test | mIoU: 87.7 -> 88.0 | Improved |
| Replace the EM M-step with a Newton-Raphson GEM update that adds a scaled skip connection | EM attention with ResNet-101 · PASCAL Context test | mIoU: 53.1 -> 54.3 | Improved |
| Replace the EM M-step with a Newton-Raphson GEM update that adds a scaled skip connection | EM attention with ResNet-101 · COCO Stuff test | mIoU: 39.9 -> 40.1 | Improved |
| Use intermediate step sizes rather than eta_train 1.0 (ablation) | HEMNet · PASCAL VOC | mIoU: 77.55 -> 78.22 | Improved |
| Increase T_train from 1 to 2 at eta_train 0.8 (ablation) | HEMNet · PASCAL VOC | mIoU: 77.26 -> 78.10 | Improved |
| Increase T_train beyond the performance peak (ablation) | HEMNet · PASCAL VOC | mIoU: 78.22 -> 77.83 | Hurt |
| Increase T_eval beyond T_train after training with eta_train 0.8 (ablation) | HEMNet · PASCAL VOC | mIoU: 78.10 -> 78.42 | Improved |
| Increase T_eval from 4 to 6 after training with eta_train 0.2 (ablation) | HEMNet · PASCAL VOC | mIoU: 77.96 -> 78.22 | No clear effect |
Closest papers by meaning
- Highway and Residual Networks learn Unrolled Iterative Estimation (ICLR 2017 · similarity 0.86)
- UNEM: UNrolled Generalized EM for Transductive Few-Shot Learning (CVPR 2025 · similarity 0.85)
- Fast and Slow Gradient Approximation for Binary Neural Network Optimization (AAAI 2025 · similarity 0.84)
- Accelerated training through iterative gradient propagation along the residual path (ICLR 2025 · similarity 0.84)
- Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization (NeurIPS 2024 · similarity 0.82)
- Training the Untrainable: Introducing Inductive Bias via Representational Alignment (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
