AI Paper Atlas

Research map / Architectures and efficiency / Quantization

INT vs. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats

Mengzhao Chen, Meng Wu, Hui Jin, Zhihang Yuan, Jing Liu, Chaoyi Zhang, Yunshui Li, Jie Huang, Jin Ma, Zeyue Xue, Zhiheng Liu, Xingyan Bin and 1 more

ICML 2026 · Architectures and efficiency › Quantization · approach: Ultra-low-bit LLM quantization

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use symmetric integer clipping range [-127,127] with BF16 scale (ablation)INT8 block-32 training · Training loss, 145Mtraining loss: 3.1354 -> 3.1251Improved
use symmetric integer clipping range with UE8M0 scale (ablation)INT8 block-32 training · Training loss, 145Mtraining loss: 3.1299 -> 3.1269Improved
use symmetric clipping at per-channel granularity with UE8M0 scale (ablation)INT8 per-channel training · Training loss, 145Mtraining loss: 3.3602 -> 3.4307Hurt
use symmetric clipping at per-channel granularity with BF16 scale (ablation)INT8 per-channel training · Training loss, 145Mtraining loss: 3.2544 -> 3.2560No clear effect
replace MXFP8 with MXINT8direct-cast inference · KL divergence x1e6, Llama-3.1-8BKL divergence: 359 -> 82Improved
replace MXFP4 with MXINT4direct-cast inference · KL divergence x1e6, Llama-3.1-8BKL divergence: 7586 -> 12380Hurt
replace NVFP4 with NVINT4 (no rotation)direct-cast inference · KL divergence x1e6, Llama-3.1-8BKL divergence: 3718 -> 4224Hurt
add random Hadamard rotation to NVINT4direct-cast inference · KL divergence x1e6, Llama-3.1-8BKL divergence: 4224 -> 3609Improved
add random Hadamard rotation to NVFP4direct-cast inference · KL divergence x1e6, Llama-3.1-8BKL divergence: 3718 -> 4752Hurt
add random Hadamard rotation to MXINT8direct-cast inference · KL divergence x1e6, Llama-3.1-8BKL divergence: 82 -> 65Improved
add random Hadamard rotation to MXFP8direct-cast inference · KL divergence x1e6, Llama-3.1-8BKL divergence: 359 -> 409Hurt
replace MXFP8 with MXINT8low-bit training 1B · Training loss, 100B tokenstraining loss: 2.6767 -> 2.6758Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy