Research map / Vision-language and multimodal / Multimodal LLMs

LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LensVLM, which renders long text into compressed images and trains a VLM (Qwen3.5-9B) to selectively expand relevant images back to source text via an Expand tool, using synthetic traces, SFT, and RL.

Problem

Visual text compression (rendering text as images for VLMs) loses readability as compression rises, so accuracy collapses at high compression rates.

Approach

Treat compression as a reversible capability: the VLM scans a compressed visual view and calls a learned Expand(k) tool to recover uncompressed source text for selected images, trained via synthetic tool-use traces, SFT, and answer-gated RL.

Key results

• At 5x preset (ECR 4.3x), LensVLM reaches 68.9% QA accuracy vs 31.3% for compressed-image reading and 72.4% full-text upper bound.
• SFT lifts accuracy at 5x from 31.3% to 63.7%; RL adds further gains to 68.9%, with selection accuracy 76.8%.
• Training reduces rendering-configuration gap from 18.0 to 0.5 points across matched configurations.

Surprising findings

• Reasoning without tool access provides no benefit over direct answers on compressed images.
• Zoom tool underperforms text tool on rendered text, but outperforms on native multimodal document images.

Limitations

• ECR is lower than nominal ICR because Expand calls append tokens; short contexts see less benefit.
• Full-text upper bound still exceeds LensVLM at all compression rates (e.g., 68.9 vs 72.4 at 5x).
• Rendered-text performance depends on the 9B backbone; smaller models (2B, 4B) are considerably weaker.

Experimental setup

Datasets: NQ HotpotQA MuSiQue HELMET PubMed Central articles (contamination study)

Benchmarks: NQ HotpotQA MuSiQue HELMET Qasper LongBench v1 RULER RepoQA CodeQueries MMLongBench-Doc

Metrics: QA accuracy selection accuracy effective compression rate latency throughput KV cache

Baselines: Text Comp. Image Base + Expand LLMLingua-2 ColPali Glyph BM25 BGE-M3 Jina-v4 Qwen3-Emb Jina-v4-MM Search-R1-7B

Builds on: DAPO Glyph LLMLingua-2 DeepEyes ColPali

Closest papers by meaning

• Efficient Large Multi-modal Models via Visual Context Compression (NeurIPS 2024 · similarity 0.89)
• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.89)
• Hybrid Token Compression for Vision-Language Models (CVPR 2026 · similarity 0.89)
• Magnifying What Matters: Attention-Guided Adaptive Rendering for Visual Text Comprehension (NeurIPS 2026 · similarity 0.89)
• LLMC+: Benchmarking Vision-Language Model Compression with a plug-and-play Toolkit (AAAI 2026 · similarity 0.89)
• Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models (CVPR 2026 · similarity 0.89)