Research map / Architectures and efficiency / Quantization
MixFP4: Enhancing NVFP4 with Adaptive FP4/INT4 Block Representations
Jiaxiang Zou, Yonghao Chen, Ruilong WU, Xinyu Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MixFP4 per-block E2M1/E1M2 selection instead of NVFP4 | NVFP4 RTN · WikiText, Llama-3.2-1B | perplexity: 13.90 -> 13.48 | Improved |
| use MixFP4 instead of NVFP4 | NVFP4 RTN · WikiText, Llama-3.1-8B | perplexity: 8.26 -> 8.06 | Improved |
| use MixFP4 instead of 4/6 | 4/6 RTN · WikiText, Qwen3-8B | perplexity: 12.56 -> 12.39 | Improved |
| use MixFP4 instead of NVFP4 on Qwen3-4B with RHT | NVFP4 +RHT · WikiText, Qwen3-4B | perplexity: 19.22 -> 18.60 | Improved |
| use MixFP4 instead of NVFP4 on Qwen3-4B without RHT | NVFP4 RTN · WikiText, Qwen3-4B | perplexity: 17.34 -> 17.42 | No clear effect |
| add random Hadamard transform to NVFP4 (ablation) | NVFP4 RTN · WikiText, Llama-3.1-8B | perplexity: 8.26 -> 8.55 | Hurt |
| add random Hadamard transform to MixFP4 (ablation) | MixFP4 RTN · WikiText, Llama-3.1-8B | perplexity: 8.06 -> 8.10 | Hurt |
| use MixFP4 with SpinQuant instead of NVFP4 | SpinQuant · Qwen3-8B avg accuracy | average accuracy: 67.20 -> 69.78 | Improved |
| use MixFP4 with GPTQ instead of 4/6 | GPTQ · Llama-3.1-8B wiki | perplexity: 8.16 -> 8.05 | Improved |
| add E1M2 format option to FP4-E2 at block size 16 (ablation) | NVFP4 · Llama-3.1-8B WikiText | perplexity: 8.26 -> 8.06 | Improved |
| add E3M0 format option to FP4-E2 at block size 16 (ablation) | NVFP4 · Llama-3.1-8B WikiText | perplexity: 8.26 -> 8.21 | Improved |
| add E1 and E3 options together at block size 16 (ablation) | NVFP4 · Llama-3.1-8B WikiText | perplexity: 8.26 -> 8.01 | Improved |
Closest papers by meaning
- RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping (NeurIPS 2026 · similarity 0.89)
- BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference (ICML 2025 · similarity 0.87)
- Normalized Architectures are Natively 4-Bit (NeurIPS 2026 · similarity 0.86)
- Quartet: Native FP4 Training Can Be Optimal for Large Language Models (NeurIPS 2025 · similarity 0.85)
- any4: Learned 4-bit Numeric Representation for LLMs (ICML 2025 · similarity 0.85)
- HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
