Research map / Vision-language and multimodal / Multimodal LLMs
Q-VLM: Post-training Quantization for Large Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Q-VLM, a post-training quantization framework for large vision-language models that mines cross-layer dependency using activation entropy as a proxy for block partitioning and optimizes the visual encoder to reduce search cost.
Problem
Post-training quantization of large vision-language models searches layer-wise rounding functions greedily, ignoring cross-layer dependency of discretization errors, which yields suboptimal quantization at low bitwidths and high search cost if done jointly.
Approach
Use activation entropy as a cheap proxy for cross-layer dependency to partition the LVLM into blocks for joint rounding-function search, and further optimize the visual encoder (entropy minimization weighted by Jacobian plus output error losses) to allow finer block decomposition and lower search cost.
Key results
- On LLaVA-7B W4A4 ScienceQA, Q-VLM reaches 79.79% accuracy vs 77.53% for QLoRA and 74.02% for AWQ (FP 89.81%).
- On LLaVA-13B the method compresses memory to 9.6G under W4A4 (vs 24.0G FP) with 80.78% accuracy, the abstract claims 2.78x memory compression and 1.44x speedup.
- Entropy proxy outperforms quantization-error proxy on LLaVA-7B W4A4 (78.66 vs 77.97) with lower search cost (25.9 vs 41.2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace layer-wise quantization with Q-VLM cross-layer dependency mining and visual encoder optimization | LLaVA-7B · ScienceQA, W4A4 | accuracy: 77.53 -> 79.79 | Helped |
| replace layer-wise quantization with Q-VLM cross-layer dependency mining and visual encoder optimization | LLaVA-13B · ScienceQA, W4A4 | accuracy: 78.64 -> 80.78 | Helped |
| add cross-layer dependency mining (ablation) | QLoRA, LLaVA-7B · ScienceQA, W4A4 | accuracy: 77.53 -> 78.66 | Helped |
| add visual encoder optimization (ablation) | QLoRA, LLaVA-7B · ScienceQA, W4A4 | accuracy: 77.53 -> 78.35 | Helped |
| use entropy instead of quantization errors as the dependency proxy (ablation) | Q-VLM, LLaVA-7B · ScienceQA, W4A4 | accuracy: 77.97 -> 78.66 | Helped |
| use entropy instead of quantization errors as the dependency proxy (ablation) | Q-VLM, LLaVA-13B · ScienceQA, W4A4 | accuracy: 78.82 -> 79.89 | Helped |
| use entropy instead of quantization errors as the dependency proxy (ablation) | Q-VLM, LLaVA-7B · ScienceQA, W4A4 | search cost: 41.2 -> 25.9 | Helped |
| increase maximum joint layer depth beyond 3 (ablation) | Q-VLM, LLaVA-v1.3-7B · ScienceQA | accuracy | No clear effect |
| quantize using Q-VLM rather than QLoRA | OpenFlamingo-3B · Vizwiz, 32 shots, 8bit | accuracy: 36.38 -> 37.60 | Helped |
| quantize using Q-VLM rather than ZeroQuant-V2 | LLaVA-7B · ScienceQA, W4A4 | inference time: 7.3h -> 6.1h | Helped |
Surprising findings
- Entropy-based proxy for cross-layer dependency yields both higher accuracy and lower search cost than using direct quantization errors as proxy.
Limitations
- Performance degrades significantly at extremely low bitwidths.
- Hardware details and compute cost are not clearly stated.
- Evaluation limited mostly to ScienceQA with small calibration set.
Experimental setup
Datasets: ScienceQA VizWiz VQA-v2 Hateful Memes
Benchmarks: ScienceQA VizWiz VQA-v2 Hateful Memes
Metrics: accuracy search cost (h) memory (G) inference time (h)
Baselines: AWQ QLoRA RPTQ ZeroQuant-V2
Builds on: AWQ QLoRA RPTQ Outlier Suppression
Closest papers by meaning
- Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients (CVPR 2026 · similarity 0.94)
- VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models (CVPR 2026 · similarity 0.92)
- MBQ: Modality-Balanced Quantization for Large Vision-Language Models (CVPR 2025 · similarity 0.91)
- Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models (NeurIPS 2026 · similarity 0.90)
- Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization (CVPR 2026 · similarity 0.90)
- Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.