Research map / Vision-language and multimodal / Multimodal LLMs

Dynamic Resolution Routing for Efficient Egocentric Grounding

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SmartRes, a pixel-space dynamic resolution routing method that uses a learned patch-level router on low-resolution features and a margin hinge loss to activate high-resolution patches for efficient egocentric visual grounding with Qwen2.5-VL.

Problem

High-resolution egocentric visual grounding is expensive because visual token processing dominates MLLM inference cost, and existing latent token pruning/merging heuristics discard object-centric spatial evidence.

Approach

Shift efficiency from post-hoc latent token pruning to proactive pixel-space resolution allocation: a low-res view guides a learned patch-level router (STE-thresholded saliency) that replaces selected LR tokens with HR token groups in raster order, with a margin hinge loss on foreground/background logit means to counter class imbalance.

Key results

• SmartRes-Lite retains 86.4% of full-resolution performance on egocentric benchmarks using 33% of visual tokens (Overall P@0.5 51.11 vs 29.23 for down-scaling at 32%).
• SmartRes-Pro retains 89.9% of full-resolution performance at 55% token budget with Overall P@0.5 53.11, and runs 1.66x faster than 50%-token Dyn-LLaVA.
• On RefCOCO-family REC, SmartRes-Pro achieves 70.88 Overall P@0.5 (80.5% retention) using 42% of tokens.

Surprising findings

• Uniform down-scaling to 50% tokens retains only ~81.8% of performance on egocentric data, while latent pruning methods like ToMe and VisionZip collapse at 50-70% retention ratios.
• Attn2HR fixed attention-based selection underperforms even random HR patch selection on FG-recall, suggesting attention heuristics are unreliable for selecting object regions.

Limitations

• Evaluated only on static images; extension to egocentric video is left as future work.
• Oracle GT-guided selection (58.45) indicates a remaining gap in routing quality.
• Only one base model (Qwen2.5-VL-3B) evaluated; generality across MLLM families is not shown.

Experimental setup

Datasets: Ego4D EgoIntention RefCOCO RefCOCO+ RefCOCOg

Benchmarks: Ego4D EgoIntention RefCOCO RefCOCO+ RefCOCOg

Metrics: P@0.5 P@0.3 mIoU FLOPs latency performance retention

Baselines: Down-scaling ToMe VisionZip FastV Dyn-LLaVA VScan Attn2HR

Builds on: Qwen2.5-VL FastV Dyn-LLaVA VisionZip

Closest papers by meaning

• When Visual Grounding Meets Gigapixel-level Large-scale Scenes: Benchmark and Approach (CVPR 2024 · similarity 0.86)
• From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.86)
• Object-Shot Enhanced Grounding Network for Egocentric Video (CVPR 2025 · similarity 0.86)
• UGround: Towards Unified Visual Grounding with Unrolled Transformers (ICML 2026 · similarity 0.85)
• GroundVLP: Harnessing Zero-Shot Visual Grounding from Vision-Language Pre-training and Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.85)
• DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.85)