Research map / Architectures and efficiency / Quantization
Bridging the Gap Between Promise and Performance for FP4 Quantization
Vage Egiazarian, Roberto L. Castro, Denis Kuznedelev, Andrei Panferov, Eldar Kurtic, Shubhra Pandit, Alexandre Noll Marques, Mark Kurtz, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Hadamard rotation (group size) to RTN (ablation) | NVFP4 RTN · W4A4 avg of 4 tasks Llama-3.1-8B | avg accuracy: 74.73 -> 74.05 | Hurt |
| add Hadamard rotation to RTN (ablation) | MXFP4 RTN · W4A4 avg of 4 tasks Llama-3.1-8B | avg accuracy: 69.32 -> 70.45 | Improved |
| replace RTN with MR-GPTQ | MXFP4 W4A4 · W4A4 avg of 4 tasks Llama-3.1-8B | avg accuracy: 69.32 -> 73.65 | Improved |
| replace GPTQ with MR-GPTQ | MXFP4 W4A4 · W4A4 avg of 4 tasks Llama-3.1-8B | avg accuracy: 70.62 -> 73.65 | Improved |
| replace GPTQ with MR-GPTQ | NVFP4 W4A4 · W4A4 avg of 4 tasks Llama-3.1-8B | avg accuracy: 75.72 -> 75.84 | No clear effect |
| add QuaRot rotation to RTN | MXFP4 W4A4 · W4A4 avg of 4 tasks Llama-3.1-8B | avg accuracy: 69.32 -> 62.90 | Hurt |
| add Hadamard to RTN (ablation) | INT4 W4A4 · W4A4 avg of 4 tasks Llama-3.1-8B | avg accuracy: 73.11 -> 74.75 | Improved |
| add Hadamard rotation to weight-only RTN (ablation) | NVFP4 W4A16 · Llama-3.1-8B W4A16 | avg accuracy: 77.37 -> 76.41 | Hurt |
| add QAT with Hadamard 32 (ablation) | MXFP4 Llama-3.2-1B · W4A4 Llama-3.2-1B | recovery %: 68.16 -> 84.55 | Improved |
| add QAT to GPTQ | NVFP4 Llama-3.1-8B · W4A4 real kernels | avg recovery: 95.7 -> 95.4 | No clear effect |
| add Hadamard 16 to RTN (ablation) | NVFP4 Llama-3.2-3B · W4A4 | recovery %: 94.45 -> 90.96 | Hurt |
| use MXINT4 with MR-GPTQ vs RTN | MXINT4 Llama-3.1-8B · W4A4 | avg accuracy: 63.05 -> 72.51 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 3 of 3 on most shared results.
- SpinQuant — SpinQuant: LLM Quantization with Learned Rotations (ICLR 2025) · better on 2 of 2 shared results
- QuaRot — QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs (NeurIPS 2024) · better on 2 of 2 shared results
- SmoothQuant — SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models (ICML 2023) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping (NeurIPS 2026 · similarity 0.91)
- Optimizing Large Language Model Training Using FP4 Quantization (ICML 2025 · similarity 0.91)
- GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models (ICML 2025 · similarity 0.89)
- Quartet: Native FP4 Training Can Be Optimal for Large Language Models (NeurIPS 2025 · similarity 0.89)
- any4: Learned 4-bit Numeric Representation for LLMs (ICML 2025 · similarity 0.89)
- ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
