Research map / Vision-language and multimodal / Multimodal LLMs

UniCompress: Token Compression for Unified Vision-Language Understanding and Generation

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UniCompress, a plug-in visual token compressor with learnable global meta tokens and an autoregressive decompressor that cuts image tokens up to 4x in unified understanding/generation models while keeping most performance.

Problem

Unified autoregressive vision-language models use long visual token sequences (e.g., 1024 tokens per image), causing high memory, training and inference cost; naive token pruning degrades generation strongly.

Approach

Insert lightweight modules around an off-the-shelf discrete tokenizer: learnable global meta-query tokens via cross-attention, average-pooling compression of local tokens, and a global-token-guided autoregressive decompressor that restores dense tokens for the image decoder; train tokenizer side first, then lightly finetune the LLM on compressed tokens.

Key results

• On GQA, UniTok drops from 55.71 to 53.07 with 4x fewer visual tokens, while generation FID rises from 16.14 to 16.33 for UniTok-Compressed.
• BAGEL-Compressed reports FID 17.22 vs 12.73 for BAGEL, and OpenUni-Compressed FID rises from 16.45 to 24.29.
• UniTok-Compressed reduces generation inference time from 32.25 to 18.96 minutes (about 41% reduction).

Surprising findings

• Compressed BAGEL reports TextVQA rising from 26.90 to 36.50 while other metrics drop, an outlier not discussed in the text.
• Reported CLIP scores are inconsistent between text and tables (e.g., UniTok-Compressed CLIP 25.0 in table vs 22.0 in text), and ablation tables do not match main results.

Limitations

• Degradation in generation FID is non-trivial for some backbones (e.g., OpenUni 16.45 to 24.29, UniFork 17.83 to 20.24).
• Fixed compression ratio; content-adaptive rates are only mentioned as future extension.
• Inconsistencies between reported numbers in text, tables and ablations reduce confidence in results.

Experimental setup

Datasets: JDB ShareGPT4V_PT ShareGPT4V MJHQ-30K

Benchmarks: GQA MME POPE Seed-bench TextVQA MMMU MM-Bench MJHQ-30K

Metrics: accuracy FID CLIPScore wall-clock training time inference time

Baselines: UniTok Vila-U VARGPT UniFork OpenUni BAGEL

Builds on: UniTok VILA-U VARGPT UniFork OpenUni BAGEL

Closest papers by meaning

• EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling (CVPR 2026 · similarity 0.91)
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.91)
• UNICOM: Unified Multimodal Modeling via Compressed Continuous Semantic Representations (NeurIPS 2026 · similarity 0.91)
• Hybrid Token Compression for Vision-Language Models (CVPR 2026 · similarity 0.91)
• Towards Unified Human Perception and Machine Understanding: Token Flow Guided Compression Framework (CVPR 2026 · similarity 0.90)
• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.90)