Research map / Probabilistic and scientific modeling / Bayesian optimization
OVLR: Efficient, Scalable, and Robust Training via Output-Level Variance-Reduced Likelihood Ratio
Minhao Zou, Tao Ren, Jinyang Jiang, Rui Tao, Zehao Li, Jiale Fu, Hui Shao, Xianhua Liu, Yijie Peng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add antithetic sampling (ablation) | OVLR, ResNet-18 · CIFAR-100 | top-1 accuracy (%): 36.62 -> 60.06 | Improved |
| add antithetic sampling (ablation) | OVLR, ResNet-18 · TinyImageNet | top-1 accuracy (%): 18.78 -> 46.90 | Improved |
| add antithetic sampling (ablation) | OVLR, ResNet-18 · CIFAR-10 | top-1 accuracy (%): 88.15 -> 88.47 | Improved |
| increase output-level repetition from n=1 to n=500 (ablation) | OVLR, ResNet-18 · CIFAR-100 | top-1 accuracy (%): 1.85 -> 62.45 | Improved |
| vary noise scale sigma from 0.1 to 5 (ablation) | OVLR with AS and n>=200 · CIFAR-100 | top-1 accuracy (%): 59.32 -> 58.98 | No clear effect |
| use Rademacher noise instead of Gaussian (ablation) | OVLR, ResNet-18 · CIFAR-10 | best accuracy (%): 87.62 -> 26.95 | Hurt |
| move perturbation from parameters to outputs (vanilla LR to OVLR) (ablation) | likelihood-ratio training · MNIST SimpleCNN | accuracy (%): 82.70 -> 96.80 | Improved |
| move perturbation from input level to output level (GLR-style to OVLR) (ablation) | likelihood-ratio training · MNIST SimpleCNN | accuracy (%): 94.60 -> 96.80 | Improved |
| replace BP with OVLR | ViT-B/16 · CIFAR-100 | accuracy (%): 86.79 -> 88.17 | Improved |
| replace BP with OVLR | ResNet-18 · CIFAR-100 | accuracy (%): 63.10 -> 62.10 | Hurt |
| optimize hard 0-1 loss with OVLR instead of CE under 60% label noise | ResNet-18 · CIFAR-10 | peak accuracy (%): 66.3 -> 73.0 | Improved |
| add pure BP gradient mixing (alpha=0.5) to OVLR (ablation) | hybrid estimator · MNIST | accuracy (%): 96.09 -> 96.20 | No clear effect |
Closest papers by meaning
- DeepZero: Scaling Up Zeroth-Order Optimization for Deep Model Training (ICLR 2024 · similarity 0.85)
- MARS: Unleashing the Power of Variance Reduction for Training Large Models (ICML 2025 · similarity 0.85)
- Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models (ICML 2025 · similarity 0.84)
- Leveraging Perturbation Robustness to Enhance Out-of-Distribution Detection (CVPR 2025 · similarity 0.84)
- Revisiting Adversarial Training at Scale (CVPR 2024 · similarity 0.84)
- OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
