Research map / Vision-language and multimodal / Multimodal LLMs
ImpQuant: Fine-Grained Importance-Aware Quantization for Large Vision-Language Models
Jundong Zhou, Tianao Cai, Yujie Huang, Xinbing Wang, Guang-Zhong Yang, Nanyang Ye
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ImpQuant, a post-training quantization approach for large vision-language models that accounts for heterogeneous information density across multimodal tokens rather than treating inputs as homogeneous sequences.
Problem
Post-training quantization of large vision-language models for low-bit deployment; existing PTQ pipelines borrowed from text-only LLMs treat multimodal inputs as homogeneous sequences, ignoring heterogeneous information density.
Approach
Fine-grained, importance-aware quantization that exploits the varying information density of different multimodal inputs instead of applying uniform LLM-style PTQ.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add token importance-aware weight quantization (W-imp) (ablation) | ImpQuant · VQA average (W4A8) | average accuracy (%): 71.38 -> 73.85 | Improved |
| add token importance-aware weight quantization (W-imp) (ablation) | ImpQuant · OCRBench (W4A8) | accuracy (%): 74.70 -> 82.90 | Improved |
| add token importance-aware weight quantization (W-imp) (ablation) | ImpQuant · TextVQA (W4A8) | accuracy (%): 77.89 -> 80.88 | Improved |
| add channel outlier-aware activation quantization (A-out) (ablation) | ImpQuant · VQA average (W4A8) | average accuracy (%): 73.33 -> 73.85 | Improved |
| add outlier-aware activation quantization (A-out) (ablation) | ImpQuant · OCRBench (W4A8) | accuracy (%): 82.30 -> 82.90 | Improved |
| 10x weight on important visual tokens in calibration loss | GPTQ-style W3A16 · VizWiz and TextVQA | accuracy (%): 67.09 -> 68.38 (VizWiz); 79.03 -> 80.64 (TextVQA) | Improved |
| 10x weight on important textual tokens in calibration loss | GPTQ-style W3A16 · VizWiz and TextVQA | accuracy (%): 67.09 -> 67.75 (VizWiz); 79.03 -> 79.96 (TextVQA) | Improved |
| ImpQuant instead of best baseline MBQ | Qwen2.5-VL-7B · VQA average (W4A8) | average accuracy (%): 69.93 -> 73.85 | Improved |
| ImpQuant instead of SmoothQuant | InternVL2-8B · VQA average (W3A16) | average accuracy (%): 71.39 -> 71.87 | Improved |
| ImpQuant instead of SmoothQuant | InternVL2-8B · POPE average accuracy (W3A16) | accuracy (%): 86.82 -> 88.18 | Hurt |
| ImpQuant instead of RTN | Qwen2.5-VL-7B · latency speedup over FP16 (batch size 1) | speedup: 1.47x -> 1.45x | No clear effect |
| ImpQuant instead of SmoothQuant | LLaVA-OneVision-7B · OCRBench (W3A16) | accuracy (%): 39.30 -> 57.20 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Q-VLM — Q-VLM: Post-training Quantization for Large Vision-Language Models (NeurIPS 2024) · better on 6 of 6 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients (CVPR 2026 · similarity 0.93)
- VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models (CVPR 2026 · similarity 0.91)
- MBQ: Modality-Balanced Quantization for Large Vision-Language Models (CVPR 2025 · similarity 0.91)
- Q-VLM: Post-training Quantization for Large Vision-Language Models (NeurIPS 2024 · similarity 0.91)
- Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.90)
- Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization (CVPR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
