Research map / Architectures and efficiency / Quantization
Be Like Water: Adaptive Floating Point for Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adaptive Floating Point 8-bit (3-bit offset + 5-bit mantissa) applied to all weights and layer outputs for inference | 13 CNNs and 3 Transformers · ImageNetV2, SST-2, MNLI, SQuAD | normalized accuracy vs FP32: 0.994 to 1.003 of FP32 accuracy | Helped |
| Use of 3-bit private offset (Auto Focus) instead of BFP with no offset at same bits (ablation) | block-based representation · ImageNet CNN weights, per-layer coverage | fraction of values not truncated to zero: min lower whisker 98% (AFP 8-bit) vs 77% (BFP 5-bit) | Helped |
| Round-to-nearest instead of truncate for mantissa (ablation) | traditional rounding for inference · ImageNet and transformer benchmarks | mantissa bits required for 99% of FP32 accuracy: max 5 bits (nearest) vs 9 bits (truncate) | Helped |
| Positive field (shared bit marking all-positive 8-value halves, reusing sign bit as mantissa bit) (ablation) | AFP shared fields · layer outputs | effective mantissa width: 35% of layer-output blocks all-positive | Helped |
| Zero field (shared bits indicating unused mantissa bit positions for offsets 0 and 1) (ablation) | AFP shared fields · ImageNet models | effective mantissa width: increases effective mantissa width for about 50% of blocks | Helped |
| AFP vs BFP at same memory: relative/absolute error reduction | AFP weights and layer outputs · ImageNet models | absolute error reduction vs BFP: 23% weights, 46% layer outputs | Helped |
| Block size 32 or 64 instead of 16 | AFP block size · ImageNet models | memory density and hardware complexity | No clear effect |
Closest papers by meaning
- Training DNNs with Hybrid Block Floating Point (NeurIPS 2018 · similarity 0.88)
- Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks (NeurIPS 2017 · similarity 0.87)
- Scaling Laws for Floating–Point Quantization Training (ICML 2025 · similarity 0.85)
- Towards Learning High-Precision Least Squares Algorithms with Sequence Models (ICLR 2025 · similarity 0.84)
- Collage: Light-Weight Low-Precision Strategy for LLM Training (ICML 2024 · similarity 0.84)
- PikeLPN: Mitigating Overlooked Inefficiencies of Low-Precision Neural Networks (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.