Research map / Vision-language and multimodal / Multimodal LLMs

Benchmarking and Enhancing VLM for Compressed Image Understanding

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces a benchmark of VLM understanding on low-bitrate compressed images across 11 codecs and 9 VLMs, decomposes the degradation into information and generalization gaps, and proposes a codec-conditioned vision-encoder adapter that recovers 10-30% performance.

Problem

Existing VLMs are evaluated on high-bitrate images, and their ability to understand heavily (low-bitrate) compressed images is unexplored; it is unclear how much of the degradation is recoverable.

Approach

Benchmark VLMs across traditional, learned, and generative codecs at low bpp, split the performance gap into an irreducible information gap and a reducible generalization gap, and close the latter with a codec- and level-conditioned vision encoder distilled from the original encoder via feature MSE.

Key results

• The adapter improves Qwen2.5-VL performance on compressed images by over 10 units on all metrics for JPEG and around 30% at low bitrate on POPE/SEEDBench.
• Generalization to unseen codecs: the JPEG-trained adapter applied to HM gives BD-POPE gain of 2.98 and BD-SEEDB 3.12 on QwenVL2.5-3B.
• Information/generalization decomposition on POPE with JPEG: performance gap 36.29, of which generalization gap 29.48 and information gap 6.81.

Surprising findings

• Increasing model size does not consistently reduce compression-induced degradation, breaking the expected scaling law.
• Generative codecs perform poorly on fine-grained OCR tasks despite better semantic reconstruction.

Limitations

• Closed-source VLMs (beyond a small GPT-4o OCRBench check) were not evaluated due to API cost.
• The information/generalization decomposition depends on the chosen adaptation strategy, so gap values are only empirical bounds.
• Adapter validated mainly on Qwen2.5-VL with a few codecs; DiffEIC generalization shows mixed results.

Experimental setup

Datasets: COCO POPE GQA COCO-Caption OCRBench MMBench MME SEED-Bench

Benchmarks: POPE GQA COCO-Caption OCRBench MMBench MME SEED-Bench

Metrics: BD-Metric accuracy CIDEr ROUGE-L BLEU bpp

Baselines: JPEG HM VTM ELIC TCM MLICpp HiFiC MS-ILLM DiffEIC RDEIC StableCodec TIC

Builds on: VLMEvalKit TransTIC RoPE

Closest papers by meaning

• LLMC+: Benchmarking Vision-Language Model Compression with a plug-and-play Toolkit (AAAI 2026 · similarity 0.91)
• VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration (ICLR 2025 · similarity 0.90)
• When MLLMs Meets Compression Distortion: A Coding Paradigm Tailored to MLLMs (ICLR 2026 · similarity 0.90)
• Bridging Compressed Image Latents and Multimodal Large Language Models (ICLR 2025 · similarity 0.89)
• VoCo-LLaMA: Towards Vision Compression with Large Language Models (CVPR 2025 · similarity 0.89)
• Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity (ICLR 2026 · similarity 0.89)