Research map / Architectures and efficiency / Quantization

LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LAQuant, a layer-wise weight-only QAT method with a one-layer lookahead loss and reasoning-domain calibration that improves reasoning accuracy of quantized LRMs without online-transform overhead.

Problem

Weight quantization of large reasoning models loses accuracy on long-decoding reasoning benchmarks even when perplexity and short-decode accuracy are preserved, and existing LRM-specific methods either add online rotation overhead or use end-to-end QAT that attenuates KV-cache fidelity.

Approach

Replace end-to-end QAT supervision with layer-wise QAT using a one-layer lookahead loss (propagating the residual through the next teacher layer) and calibrate on Hessian-aligned reasoning traces (DeepSeek-R1 on OpenR1-Math), yielding standard integer weights with no online transform.

Key results

• On Qwen3-4B W3G128, LAQuant reaches 54.69 AIME25 Pass@1 vs 39.58 for ParoQuant (+15.11 pp) and 52.76 for ParoQuant++, with 3.42x decoding speedup over FP16 on RTX A6000 vs ParoQuant's 3.01x.
• On Qwen3-1.7B W3G128, LAQuant averages 36.60 across six reasoning tasks vs 34.54 for ParoQuant++ and 24.57 for ParoQuant.
• LAQuant matches ParoQuant on WikiText2/C4 perplexity and zero-shot accuracy while avoiding online transformations.

Surprising findings

• Under matched calibration, ParoQuant++ (reasoning data) dramatically outperforms default ParoQuant, e.g. R1-Distill-Llama-8B W3 AIME25 rises from 0.05 to 25.57, indicating calibration data dominates for LRMs.
• Controlled E2E supervision yields lower top-1 agreement with the BF16 teacher (88.44%) than layer-wise supervision, contrary to the expectation that logit-matching should help.

Limitations

• Gains over ParoQuant on general tasks are negligible; LAQuant only matches perplexity and zero-shot accuracy.
• Some reasoning results are not best (e.g., MMLU-Pro at W3 Qwen3-4B and W4 Qwen3-8B, GPQA on some models).
• ReasoningQAT compared under its prescribed public configuration with different calibration data and larger token budget, so comparison is not fully controlled.

Experimental setup

Datasets: OpenR1-Math-220k WikiText2 C4 RedPajama OpenThoughts3

Benchmarks: AIME24 AIME25 GPQA Diamond LSAT-AR MMLU-Pro LiveCodeBench WikiText2 C4 ARC-Challenge ARC-Easy BoolQ HellaSwag

Metrics: Pass@1 perplexity zero-shot accuracy decoding throughput (tokens/sec) RMSE

Baselines: GPTQ ReasoningQAT ParoQuant ParoQuant++ AWQ

Builds on: EfficientQAT ParoQuant GPTQ AWQ

Closest papers by meaning

• When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models (ICLR 2026 · similarity 0.89)
• ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scaling (NeurIPS 2026 · similarity 0.88)
• Efficient Reasoning via Constrained Optimization in Latent Space (NeurIPS 2026 · similarity 0.88)
• SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning (ICML 2026 · similarity 0.88)
• SpinQuant: LLM Quantization with Learned Rotations (ICLR 2025 · similarity 0.87)
• BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference (ICML 2026 · similarity 0.87)