Research map / Architectures and efficiency / Quantization

ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ConQuR, a post-training rotation calibration that aligns normalized activations with inscribed-hypercube corners, solved via closed-form Procrustes updates in an online mini-batch scheme for W4A4 LLM quantization.

Problem

Low-bit weight-activation quantization of LLMs is hurt by activation outliers; existing learned rotation methods either need expensive end-to-end training or store large activation corpora for calibration, and calibration is decoupled from quantized inference.

Approach

Learn orthogonal rotations that push normalized activations toward hypercube corners (equal-magnitude coordinates), which is equivalent to maximizing the l1 norm; the alternating sign-target/Procrustes updates give closed-form rotation updates, applied online over mini-batches with quantized activations so no activation corpus is stored.</part>

Key results

• On Llama-3 8B under 4-4-16, quantization-aware online calibration reaches average PPL 10.62 vs 10.79 for DartQuant and 11.06 for QuaRot, with 9-task average 62.51.
• Calibration takes about 0.42 GPU-hours on a single RTX A6000 with no offline activation storage, versus 32 GPU-hours for SpinQuant and ~193GB of stored activations for DartQuant (Llama-2 7B).
• On Llama-3 70B under 4-4-16, full-precision-calibrated variant achieves average PPL 8.25, versus 8.31 for DartQuant.

Surprising findings

• Offline stored-activation variant performs comparably to the online method when calibration is full precision, suggesting online updates approximate i.i.d. optimization over stored activations.
• Gains from quantization-aware calibration are most consistent in PPL, while zero-shot accuracy differences among top methods are small.

Limitations

• Calibration performance may depend on the calibration data distribution.
• Evaluation focuses on perplexity and common downstream benchmarks; generation quality and latency are not broadly evaluated.
• The theoretical link to asymmetric quantization holds only up to vanishing dimensional corrections.

Experimental setup

Datasets: WikiText-2

Benchmarks: WikiText-2 PTB C4 WinoGrande SocialIQA LAMBADA MMLU ARC-Easy ARC-Challenge HellaSwag OpenBookQA PIQA

Metrics: perplexity zero-shot accuracy normalized participation ratio relative squared quantization error

Baselines: QuaRot SpinQuant DartQuant DFRot

Builds on: QuaRot SpinQuant DartQuant DFRot QuIP#

Closest papers by meaning

• SpinQuant: LLM Quantization with Learned Rotations (ICLR 2025 · similarity 0.90)
• Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge (AAAI 2024 · similarity 0.90)
• Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference (ICLR 2025 · similarity 0.89)
• La RoSA: Enhancing LLM Efficiency via Layerwise Rotated Sparse Activation (ICML 2025 · similarity 0.89)
• Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.88)
• ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization (AAAI 2025 · similarity 0.88)