Research map / Architectures and efficiency / Quantization
Is Finer Better? The Limits of Microscaling Formats in Large Language Models
Andrea Fasoli, Monodeep Kar, Chi-Chun Liu, Swagath Venkataramani, Viji Srinivasan, Leland Chang, Naigang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FP8 UE5M3 scales instead of UE4M3 (ablation) | FP4 microscaling, granite-3.3-8b · Wikitext2 perplexity (FP4, block size 8) | perplexity: 7.43 -> 5.04 | Improved |
| use FP8 UE5M3 scales instead of UE4M3 (ablation) | FP4 microscaling, bamba-9b-v2 · Wikitext2 perplexity (FP4, block size 8) | perplexity: 21.25 -> 6.53 | Improved |
| use FP8 UE5M3 scales instead of UE4M3 (ablation) | FP4 microscaling, granite-3.3-8b · GSM8K (FP4, block size 8) | accuracy: 32.37 -> 56.17 | Improved |
| use FP8 UE5M3 scales instead of UE4M3 (ablation) | FP4 microscaling, bamba-9b-v2 · GSM8K (FP4, block size 8) | accuracy: 2.65 -> 39.42 | Improved |
| use FP8 UE5M3 scales instead of UE4M3 (ablation) | FP4 microscaling, nemotron-nano-9b-v2 · GSM8K (FP4, block size 8) | accuracy: 72.71 -> 77.71 | Improved |
| use FP8 UE5M3 scales instead of UE4M3 (ablation) | FP4 microscaling, llama-3.1-8b · Wikitext2 perplexity (FP4, block size 8) | perplexity: 7.23 -> 6.79 | Improved |
| add dynamic per-tensor scaling to UE4M3 scales (ablation) | FP4 microscaling, granite-3.3-8b · GSM8K (FP4, block size 8) | accuracy: 32.37 -> 44.88 | Improved |
| add dynamic per-tensor scaling to UE4M3 scales (ablation) | FP4 microscaling, bamba-9b-v2 · Wikitext2 perplexity (FP4, block size 8) | perplexity: 21.25 -> 6.53 | Improved |
| use UE5M3 without per-tensor scaling instead of UE4M3 with per-tensor scaling (ablation) | FP4 microscaling, granite-3.3-8b · GSM8K (FP4, block size 8) | accuracy: 44.88 -> 56.17 | Improved |
| use UE5M3 without per-tensor scaling instead of UE4M3 with per-tensor scaling (ablation) | FP4 microscaling, llama-3.1-8b · GSM8K (FP4, block size 8) | accuracy: 43.21 -> 42.15 | No clear effect |
| use UE5M3 without per-tensor scaling instead of UE4M3 with per-tensor scaling (ablation) | FP4 microscaling, bamba-9b-v2 · GSM8K (FP4, block size 8) | accuracy: 40.41 -> 39.42 | No clear effect |
| use UE5M3 without per-tensor scaling instead of UE4M3 with per-tensor scaling (ablation) | FP4 microscaling, nemotron-nano-9b-v2 · GSM8K (FP4, block size 8) | accuracy: 78.92 -> 77.71 | No clear effect |
Closest papers by meaning
- What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation (AAAI 2024 · similarity 0.87)
- ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs (NeurIPS 2026 · similarity 0.87)
- SqueezeLLM: Dense-and-Sparse Quantization (ICML 2024 · similarity 0.87)
- BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference (ICML 2025 · similarity 0.87)
- RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping (NeurIPS 2026 · similarity 0.86)
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
