Research map / Vision-language and multimodal / Multimodal LLMs

Skip-It? Theoretical Conditions for Layer Skipping in Vision–Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an information- and learning-theoretic redundancy framework (geometric, proximal, functional, informational) that gives label-free conditions for vision-token layer skipping (late entry / early exit) in VLMs, validated on LLaVA and Qwen2.5-VL.

Problem

Layer skipping methods for vision-language models rely on ad-hoc heuristics or hyperparameter sweeps; there is no principled criterion for when skipping layers or vision tokens preserves performance.

Approach

Defines geometric, proximal, functional, and informational notions of redundancy between adjacent layer/token representations, proves implications among them (e.g., cosine closeness bounds functional and information-theoretic differences), unifies layer skipping into early exit and late entry of vision tokens, and uses label-free calibration statistics (cosine distance, probability of proximity, v

Key results

• Early and late vision tokens in LLaVA 1.5 7B/13B and Qwen2.5-VL show low adjacent-layer cosine distance and high proximal-redundancy probability (e.g., P[D_V<0.05] ~0.97 at layer 4 for LLaVA 1.5 7B), consistent with prior findings.
• Late entry skipping of vision tokens up to layer 4 keeps General VQA accuracy close to baseline for LLaVA 1.5 7B (0.553 vs 0.564), while skipping to layer 8 drops it to 0.370.
• Early exit of vision tokens at layer 28 keeps accuracy near baseline across models (e.g., LLaVA 1.5 7B General VQA 0.581 vs 0.582), with degradation when exiting earlier where visual attention ratio is high.

Surprising findings

• Redundancy conditions depend on task: textual redundancy in early layers is low for discriminative VQA but high for captioning.
• Multimodal reasoning accuracy is near chance for several datasets, so redundancy metrics show little degradation signal there.

Limitations

• Theoretical bounds rely on Lipschitz and boundedness assumptions that are only estimated empirically via spectral norms.
• Redundancy metrics do not translate into measured speedups; the paper reports accuracy only.
• Experiments are limited to a few VLM families and up to 13B parameters.

Experimental setup

Datasets: GQA VQA Visual7W AI2D OCRBench TextVQA MMMU RealWorldQA MMStar MathVision COCO Flickr30k

Benchmarks: GQA VQA Visual7W AI2D OCRBench TextVQA MMMU RealWorldQA MMStar MathVision COCO captioning Flickr30k captioning

Metrics: accuracy BLEU CIDEr SPICE cosine distance proximal redundancy probability visual attention ratio

Baselines: VTW LayerSkip DeepInsert SAINT FastV

Builds on: LayerSkip VTW DeepInsert FastV SAINT

Closest papers by meaning

• Each Complexity Deserves a Pruning Policy (NeurIPS 2025 · similarity 0.90)
• FlowCut: Rethinking Redundancy via Information Flow for Efficient Vision-Language Models (NeurIPS 2025 · similarity 0.90)
• Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves (CVPR 2025 · similarity 0.89)
• SwiftVLM: Efficient Vision-Language Model Inference via Cross-Layer Token Bypass (NeurIPS 2026 · similarity 0.88)
• Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.88)
• Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoders (ICLR 2026 · similarity 0.88)