AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Efficient Segmentation with Multimodal Large Language Model via Token Routing

Changsong Wen, Zelin Peng, Yu Huang, Wei Shen

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an efficient MLLM-based segmentation method that routes tokens to reduce LLM computation, rather than pruning visual tokens.

Problem

The LLM component of MLLMs is computationally expensive for open-world segmentation, where efficiency is a central concern; existing efficient MLLM methods mainly prune visual tokens.

Approach

Uses token routing to reduce LLM computation for segmentation instead of pruning visual tokens, as an alternative to existing visual token pruning approaches.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
route mask tokens only from layer 13 of the LLM (skip early layers) (ablation)PSALM (Phi-1.5) · RefCOCOcIoU: 81.56 -> 81.65No clear effect
delay mask token insertion further to layer 18 (ablation)Token Routing · RefCOCOcIoU: 81.65 -> 80.48Hurt
skip the LLM entirely for mask tokens (ablation)Token Routing · RefCOCOcIoU: 81.65 -> 79.93Hurt
skip FFN for all image tokens (ablation)PSALM (Phi-1.5) · RefCOCOcIoU: 81.71 -> 80.40Hurt
add learnable gate to proxy-guided update (ablation)Proxy-guided image routing · RefCOCOcIoU: 81.02 -> 81.71Improved
increase proxy tokens from 8 to 32 (ablation)Proxy-guided image routing · RefCOCOcIoU: 81.71 -> 81.65No clear effect
prune image tokens (FastV, 50%) instead of routingPSALM (Phi-1.5) · RefCOCO series averagecIoU: 69.2 -> 66.9Hurt
apply late-stage mask token routing alone (ablation)PSALM (Phi-1.5) · RefCOCO series averageFLOPs (G): 1126.98 -> 998.32Improved
apply proxy-guided image token routing alone (ablation)PSALM (Phi-1.5) · RefCOCO series averageFLOPs (G): 1126.98 -> 758.75Improved
combine both routing strategiesPSALM (Phi-1.5) · RefCOCO series averagelatency (ms): 123 -> 83Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy