Research map / Architectures and efficiency / Quantization
Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use S2FP8 instead of FP32 | ResNet-20/34/50 · CIFAR-10 validation | accuracy: 0.4; 0.5; -0.2 | Mixed |
| Use FP8 without loss scaling instead of S2FP8 | ResNet-20/34/50 · CIFAR-10 validation | accuracy: 91.1 vs 17.9; 92.0 vs 13.5; 93.2 vs 11.5 | Hurt |
| Use S2FP8 instead of FP32 | ResNet-18/50 · ImageNet-1k validation | accuracy: -0.7; -1.0 | Hurt |
| Use FP8 without loss scaling instead of S2FP8 | ResNet-18/50 · ImageNet-1k validation | accuracy: NaN | Hurt |
| Use FP8 with tuned exponential loss scaling instead of S2FP8 | Transformer tiny · English-Vietnamese | BLEU: 25.3 -> 21.3 | Hurt |
| Use FP8 without loss scaling instead of S2FP8 | Transformer tiny · English-Vietnamese | BLEU: NaN | Hurt |
| Use S2FP8 instead of FP32 | NCF · MovieLens-1M | Hit Ratio: 0.666 -> 0.663 | Hurt |
| Use FP8 without loss scaling instead of S2FP8 | NCF · MovieLens-1M | Hit Ratio: 0.663 -> 0.633 | Hurt |
| Use FP8 with constant loss scaling of 100 instead of S2FP8 | ResNet-20/34/50 · CIFAR-10 validation | accuracy: 91.1 vs 91.1; 92.0 vs 92.0; 93.2 vs 92.9 | No clear effect |
| Use FP8 with loss scaling and keep first and last layers in FP32 | ResNet-50 · ImageNet-1k validation | accuracy: 75.2 vs 75.3 | Hurt |
Closest papers by meaning
- Hybrid 8-bit Floating Point (HFP8) Training and Inference for Deep Neural Networks (NeurIPS 2019 · similarity 0.93)
- Training Deep Neural Networks with 8-bit Floating Point Numbers (NeurIPS 2018 · similarity 0.92)
- FP8 Quantization: The Power of the Exponent (NeurIPS 2022 · similarity 0.90)
- Scaling Laws for Floating–Point Quantization Training (ICML 2025 · similarity 0.86)
- Scaling FP8 training to trillion-token LLMs (ICLR 2025 · similarity 0.85)
- µnit Scaling: Simple and Scalable FP8 LLM Training (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.