Research map / Architectures and efficiency / Quantization

RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: KV cache quantization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RaZeR, an enhanced NVFP4 format that repurposes the redundant FP4 negative zero and spare block-scale bits as per-block special quantization values, improving 4-bit LLM weight-only and weight-activation quantization at the same footprint, with GPU kernels and a modified tensor core.

Problem

NVFP4 4-bit LLM quantization wastes representational capacity: FP4 has a redundant negative zero and the FP8 block scale has an unused sign bit, limiting accuracy for weight-only and weight-activation quantization.

Approach

Remap the redundant FP4 zero to a per-block special value (e.g., ±5, plus extra values for weights) selected by metadata stored in the redundant bits of the FP8 block scale (E3M3 scale for weights frees 1 bit, sign bit gives 1 bit for activations), keeping the same 4.5-bit footprint.

Key results

• RaZeR reduces average perplexity loss relative to NVFP4 by 34.6% (weight-only) and 31.2% (weight-activation).
• Versus FourOverSix, RaZeR reduces average perplexity loss by 29.2% (weight-only) and 23.3% (weight-activation).
• Modified RaZeR tensor core adds 3.7% area and 13.5% power over the baseline NVFP4 tensor core, about 0.37% chip area at system level.

Surprising findings

• Weights tolerate a reduced 6-bit E3M3 block scale with no perplexity loss versus E4M3, while activations do not.

Limitations

• Activation special values require online double quantization per block, adding some runtime overhead.
• Weight-activation GEMM on Blackwell tensor cores lacks native remap support, requiring a two-pass method.
• Evaluation focuses on Llama and Qwen3 models up to 32B; the hardware tensor core is only modeled, not fabricated.

Experimental setup

Datasets: Pile (calibration for activation special values)

Benchmarks: WikiText-2 C4 PIQA HellaSwag Winogrande ARC LAMBADA GSM8K MMLU

Metrics: perplexity zero-shot accuracy reasoning accuracy decode throughput kernel latency area power

Baselines: MXFP4 NVFP4 GPTQ AWQ SqueezeLLM 4over6 NF4 BlockDialect MR-GPTQ Any-Precision-LLM Marlin Marlin-FP4

Builds on: NVFP4 MXFP4 GPTQ AWQ FourOverSix MR-GPTQ Marlin

Closest papers by meaning

• Optimizing Large Language Model Training Using FP4 Quantization (ICML 2025 · similarity 0.89)
• BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference (ICML 2025 · similarity 0.89)
• Normalized Architectures are Natively 4-Bit (NeurIPS 2026 · similarity 0.88)
• any4: Learned 4-bit Numeric Representation for LLMs (ICML 2025 · similarity 0.88)
• ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals (ICML 2025 · similarity 0.87)
• ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.87)