Research map / Large language models / Language, safety and interpretability
Block Rotation is All You Need for MXFP4 Quantization
Yuantian Shao, Peisong Wang, Yuanteng Chen, Chang Xu, Zhihui Wei, Jian Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply random Hadamard rotation (QuaRot) before RTN under MXFP4 | RTN MXFP4 · LLaMA-2 7B WikiText2 / avg accuracy | perplexity / accuracy: 7.08/57.26 -> 13.09/50.32 | Hurt |
| replace global rotation with block-wise rotation (BRQ) (ablation) | QuaRot+ MXFP4 · LLaMA-3.2 1B WikiText2 / avg accuracy | perplexity / accuracy: 12.78/48.83 -> 11.95/49.87 | Improved |
| replace global rotation with block-wise rotation (BRQ) (ablation) | QuaRot+ MXFP4 · Qwen2.5-1.5B WikiText2 / avg accuracy | perplexity / accuracy: 12.80/53.50 -> 12.15/54.83 | Improved |
| replace global rotation with block-wise rotation (BRQ) (ablation) | QuaRot+ MXFP4 · LLaMA-3 8B WikiText2 | perplexity: 7.68 -> 7.14 | Improved |
| replace global rotation with block-wise rotation (BRQ) (ablation) | QuaRot+ MXFP4 · LLaMA-2 70B WikiText2 | perplexity: 3.76 -> 3.62 | Improved |
| use random block rotation (BRQ) instead of optimized global rotation (SpinQuant) (ablation) | SpinQuant MXFP4 · LLaMA-3 8B WikiText2 / avg accuracy | perplexity / accuracy: 7.62/61.93 -> 7.14/63.54 | Improved |
| optimize block rotation matrices within SpinQuant (BRQ_Spin) | BRQ · LLaMA-3.2 3B WikiText2 | perplexity: 9.41 -> 9.08 | Improved |
| optimize block rotation matrices within SpinQuant (BRQ_Spin) | BRQ · LLaMA-3 8B WikiText2 | perplexity: 7.14 -> 7.13 | No clear effect |
| use rotation dimension equal to MXFP4 block size (32) instead of larger or smaller (ablation) | BRQ · LLaMA-3 / 3.2 WikiText2 | perplexity | Improved |
| add GPTQ compensation to RTN under MXFP4 | RTN MXFP4 · LLaMA-3.2 1B WikiText2 / avg accuracy | perplexity / accuracy: 15.91/46.89 -> 13.35/48.52 | Improved |
| use block-wise rotation instead of global rotation for online rotation (ablation) | QuaRot · LLaMA-2 7B prefill latency (batch 1, SeqLen 512) | overhead: 7.03% -> 4.22% | Improved |
| apply SmoothQuant before RTN under MXFP4 | RTN MXFP4 · LLaMA-3.2 1B WikiText2 | perplexity: 15.91 -> 16.86 | Hurt |
Closest papers by meaning
- BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference (ICML 2025 · similarity 0.90)
- RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping (NeurIPS 2026 · similarity 0.89)
- Optimizing Large Language Model Training Using FP4 Quantization (ICML 2025 · similarity 0.89)
- Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference (ICLR 2025 · similarity 0.89)
- ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.89)
- CBQ: Cross-Block Quantization for Large Language Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
