Research map / Architectures and efficiency / Quantization

SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SPHQuant, a rotation-free spherical weight-only PTQ method that decomposes 8D weight vectors into signs, radius and a positive-direction codebook, with extra radius bits and angular codebook fine-tuning plus a fused GEMV kernel, for extreme low-bit VLM quantization.

Problem

Weight-only post-training quantization of vision-language models to 2-3 bits suffers from weight outliers in Cartesian coordinates and existing methods either degrade or add rotation/runtime overhead.

Approach

Represent each 8D weight vector in spherical form (signs, radius, positive unit direction), isolate outlier magnitude in the radius and give the radius extra bits, quantize directions with a small sign-separated codebook fine-tuned via hyperspherical angles.

Key results

• At W2 on Qwen3-VL-8B-Instruct, SPHQuant averages 63.0 vs QTIP 62.9 across five benchmarks, with 2.25 encoded bits per weight.
• SPHQuant improves decoding throughput over QTIP by 30.3% on RTX A6000 (132.96 vs 102.05 tokens/s) for Qwen3-VL-8B at W2.
• At W3 on Qwen3.5-9B, SPHQuant averages 65.9 vs QTIP 65.2 and ParoQuant 63.7; on MMMU-Pro with thinking, 61.9 vs QTIP 61.2 at W3.

Surprising findings

• On RTX 4090, plain RTN achieves higher decoding throughput (300.06 tokens/s) than SPHQuant (288.63 tokens/s).
• At W2 on Qwen3.5-9B, SPHQuant (62.2) slightly underperforms QTIP (62.4) on average accuracy.

Limitations

• Throughput evaluated only on RTX A6000 and RTX 4090; not on embedded or mobile devices.
• Effective payload is 2.25 bits at W2 (0.25 more than strict 2-bit methods), and extra radius storage matters near memory limits.
• Evaluation limited to batch size 1 and fixed sequence lengths; benchmarks not covering real-world deployment tasks.

Experimental setup

Datasets: COCO

Benchmarks: MMMU_val TextVQA ScienceQA RealWorldQA MMStar MMMU-Pro

Metrics: accuracy decoding throughput (tokens/s) GPU memory (GB) validation loss

Baselines: RTN AWQ MBQ ParoQuant QTIP PVQ PCDVQ

Builds on: QTIP ParoQuant AWQ GPTQ

Closest papers by meaning

• VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models (CVPR 2026 · similarity 0.90)
• Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients (CVPR 2026 · similarity 0.90)
• Zero-Shot Quantization via Weight-Space Arithmetic (NeurIPS 2026 · similarity 0.89)
• QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Models (NeurIPS 2025 · similarity 0.89)
• Q-VLM: Post-training Quantization for Large Vision-Language Models (NeurIPS 2024 · similarity 0.89)
• Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models (NeurIPS 2026 · similarity 0.89)