Research map / Architectures and efficiency / Quantization
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Progressive Fractional Quantization (PFQ) instead of static precision (ablation) | SBM ResNet-18 · ImageNet | accuracy (%): 69.51 -> 69.68 | Helped |
| add PFQ to Transformer-base (ablation) | SBM · WikiText-103 | perplexity: 31.55 -> 31.49 | Helped |
| replace static FW-8/BW-16 with PFQ on MobileNetV2 (ablation) | SBM · CIFAR-10 | accuracy (%): 93.69 (+0.03%) | No clear effect |
| PFQ schedule FW(4,4,6,8)/BW(6,8,10,12) on MobileNetV2 (ablation) | SBM · CIFAR-100 | accuracy (%): 75.07 (+0.37%) | Helped |
| add Dynamic Fractional Quantization (DFQ) for input-dependent per-layer precision (ablation) | SBM ResNet-38 · CIFAR-10 | MACs: 1.18E+14 -> 4.96E+13 | Helped |
| combine PFQ and DFQ into FracTrain (ablation) | DFQ ResNet-38 · CIFAR-10 | MACs: 4.96E+13 -> 3.97E+13 | Helped |
| combine PFQ and DFQ into FracTrain (ablation) | DFQ ResNet-74 · CIFAR-100 | accuracy (%): 70.58 -> 71.03 | Helped |
| FracTrain vs SBM energy | SBM ResNet-74 · CIFAR-10 | energy (kJ): 75.01 -> 33.40 | Helped |
| FracTrain vs SBM latency | SBM ResNet-74 · CIFAR-10 | latency (min): 692.29 -> 321.98 | Helped |
| FracTrain on ResNet-74 CIFAR-10 accuracy | SBM · CIFAR-10 | accuracy (%): 93.04 -> 92.97 | No clear effect |
| FracTrain on ResNet-18 | SBM · ImageNet | accuracy (%): 69.51 -> 69.44 | No clear effect |
| FracTrain on ResNet-34 | SBM · ImageNet | accuracy (%): 73.34 -> 73.03 | Hurt |
Closest papers by meaning
- Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference (AAAI 2020 · similarity 0.86)
- Don’t Waste Your Bits! Squeeze Activations and Gradients for Deep Neural Networks via TinyScript (ICML 2020 · similarity 0.86)
- Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators (ICLR 2024 · similarity 0.85)
- Stepping Forward on the Last Mile (NeurIPS 2024 · similarity 0.84)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.84)
- LOTION: Smoothing the Optimization Landscape for Quantized Training (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.