Research map / Optimization / Optimizers
Low-Precision Stochastic Gradient Langevin Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SGLD instead of SGD with low-precision gradient accumulators | 8-bit fixed-point models · CIFAR-100 | test error (%): 28.86 -> 26.62 | Helped |
| use SGLD instead of SGD with low-precision gradient accumulators | 8-bit block-floating-point models · CIFAR-100 | test error (%): 26.19 -> 25.22 | Helped |
| use SGLD instead of SGD with full-precision gradient accumulators | 8-bit block-floating-point models · ImageNet | Top-1 test error: 32.23 -> 31.47 | Helped |
| use SGLD instead of SGD in full precision | ImageNet ResNet-18 · ImageNet | Top-1 test error: 30.56 -> 30.39 | Helped |
| add variance-corrected quantization to low-precision SGLD (ablation) | naive SGLDLP-L · CIFAR-100, 8-bit fixed point | test error (%): 27.25 -> 26.62 | Helped |
| add variance-corrected quantization to low-precision SGLD (ablation) | naive SGLDLP-L · CIFAR-100, 8-bit block floating point | test error (%): 26.38 -> 25.22 | Helped |
| add variance-corrected quantization to low-precision SGLD (ablation) | naive SGLDLP-L · CIFAR-10, 8-bit block floating point | test error (%): 5.85 -> 5.51 | Helped |
| use deterministic rather than stochastic weight rounding (ablation) | SGLDLP-L · CIFAR-10, 8-bit block floating point | test error (%): 5.51 -> 7.37 | Hurt |
| use low-precision gradient accumulators without variance correction (ablation) | SGLDLP-L · standard Gaussian target, small stepsizes | convergence to target distribution | Hurt |
| use variance-corrected quantization instead of naive low-precision quantization (ablation) | SGLDLP-L · standard Gaussian target, varying stepsizes | convergence to target distribution | Helped |
| quantize activations to low precision (ablation) | 8-bit fixed-point deep-learning experiments · CIFAR and IMDB | test error (%) | Hurt |
Closest papers by meaning
- The promises and pitfalls of Stochastic Gradient Langevin Dynamics (NeurIPS 2018 · similarity 0.91)
- Time-independent Generalization Bounds for SGLD in Non-convex Settings (NeurIPS 2021 · similarity 0.91)
- Flatness-Aware Stochastic Gradient Langevin Dynamics (ICML 2026 · similarity 0.90)
- Generalization of noisy SGD in unbounded non-convex settings (ICML 2025 · similarity 0.88)
- Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise (NeurIPS 2025 · similarity 0.87)
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.