Research map / Architectures and efficiency / Quantization

RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models

NeurIPS 2025 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RSAVQ, a vector quantization framework for LLM weights that uses Fisher-information (Riemannian) geometry to project quantization error onto low-sensitivity natural-gradient directions and to allocate per-channel bits by curvature energy.

Problem

Low-bit (2-4 bit) post-training quantization of LLMs with vector quantization suffers from unconstrained quantization error direction and uniform, sensitivity-agnostic bit allocation across weight channels.

Approach

Models the weight space as a Riemannian manifold with the Fisher information metric; EDSG projects VQ error onto negative natural gradient directions, and WCSG scores each channel's Riemannian curvature energy to allocate bits proportional to log2 of that score, grouped for shared codebooks.

Key results

• On LLaMA-3 8B at 2 bits, RSAVQ reports WikiText-2 PPL 8.79 vs VPTQ 9.29 and 61.72 zero-shot average vs 60.22.
• On LLaMA-2 7B at 2 bits, the k-means baseline reaches 9.20 PPL; adding EDSG gives 7.29 and adding WCSG gives 5.81 (ablation table).
• At 2 bits on LLaMA-2 7B, RSAVQ is 5.97 PPL vs VPTQ 6.13 in the main table, but the paper's own ablation reports 5.81 for the same setting, an internal inconsistency.

Measured effects reported in the paper

Surprising findings

• The paper's ablation and main tables report different PPL values (5.81 vs 5.97) for the same LLaMA-2 7B 2-bit setting.

Limitations

• Validated mainly on LLaMA-family Transformers; limited multimodal, vision, and RL evaluation.
• Scalar quantization extension unexplored.
• VQ codebook lookups add memory-access overhead on heterogeneous hardware.

Experimental setup

Datasets: RedPajama WikiText-2

Benchmarks: WikiText-2 WinoGrande HellaSwag PIQA ARC-e ARC-c

Metrics: perplexity zero-shot accuracy tokens/s memory (GB)

Baselines: GPTQ GPTVQ AQLM DB-LLM QuIP QuIP# VPTQ

Builds on: product quantization VPTQ natural gradient K-FAC rate-distortion theory

Closest papers by meaning

• QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models (ICLR 2024 · similarity 0.89)
• ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.89)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models (ICML 2025 · similarity 0.89)
• Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.89)
• GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling (NeurIPS 2026 · similarity 0.89)
• Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.88)