Research map / Vision-language and multimodal / Multimodal LLMs

Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes QIG, a token-level post-training quantization method that uses quantization-aware integrated gradients (with IQR clipping) to weight per-token reconstruction error during scale search for LVLMs.

Problem

Existing LVLM post-training quantization measures token sensitivity only at the modality level, missing cross-token interactions and token-level quantization error, causing a gap to full-precision accuracy.

Approach

Adapts integrated gradients to attribute the quantization-induced output discrepancy (f(x,w) - f(x,w^q)) from the quantized input baseline to each token, then uses IQR-clipped, normalized scores to reweight the channel-wise equalization reconstruction loss.

Key results

• Under W3A16, QIG raises LLaVA-onevision-7B average accuracy to 72.04% vs. 70.44% for MBQ and 73.37% FP16 (gap 1.33%).
• Under W4A8 on InternVL2-8B, QIG averages 72.04% vs. 71.38% for MBQ (FP16 74.31%).
• Integrating QIG into GPTQ improves InternVL2-8B W3A16 ChartQA from 76.40% to 78.12%.

Surprising findings

• Gradient-based token-level weighting performs worse than modality-level weighting (55.78 vs 57.36 on VizWiz), and generic LLM PTQ methods such as GPTQ and SmoothQuant often underperform naive RTN on LVLMs.

Limitations

• Gains over MBQ are modest (about 0.5% average) and some configurations show near-zero gains (e.g., W4A8 LLaVA-onevision 70.16 vs 70.23).
• Completeness property may be broken by IQR clipping post-processing, as the paper acknowledges.
• Calibration limited to 128 COCO-style caption samples with mostly single-GPU experiments; results on wider model families are limited.

Experimental setup

Datasets: ShareGPT4V COCO Caption (calibration) InfoVQA (OCR calibration ablation)

Benchmarks: VizWiz MMMU ChartQA AI2D ScienceQA DocVQA OCRBench PIQA ARC-e ARC-c MMLU PPL

Metrics: accuracy perplexity

Baselines: RTN GPTQ AWQ MBQ SmoothQuant

Builds on: Integrated Gradients MBQ AWQ GPTQ SmoothQuant

Closest papers by meaning

• Q-VLM: Post-training Quantization for Large Vision-Language Models (NeurIPS 2024 · similarity 0.94)
• VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models (CVPR 2026 · similarity 0.93)
• MBQ: Modality-Balanced Quantization for Large Vision-Language Models (CVPR 2025 · similarity 0.92)
• Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models (NeurIPS 2026 · similarity 0.92)
• Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization (CVPR 2026 · similarity 0.92)
• Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.91)