Research map / Vision-language and multimodal / Multimodal LLMs

When and What to Prune? Stage-Aware Visual Token Pruning for Efficient VLA

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SAPrune, a training-free visual token pruning method for VLA models that selects pruning layers from calibrated action-to-visual attention dynamics and applies a dual-path (attention-core plus functional-diversity) retention rule, pruning 87.5% of visual tokens with up to 1.718x speedup.

Problem

VLA inference is expensive because many visual tokens pass through deep Transformer layers, and existing pruning methods use fixed pruning schedules and static importance or diversity criteria that may discard action-relevant tokens under aggressive reduction.

Approach

Choose pruning layers adaptively from rollout-averaged top-k action-to-visual attention mass dynamics (plateau and redistribution detection on calibration rollouts), and at each layer retain tokens via a dual-path rule: top action-attention tokens plus query-conditioned functional-signature coverage tokens that reduce redundancy.

Key results

• On LIBERO with OpenVLA-OFT, SAPrune keeps 32 of 256 visual tokens, reaching 94.5% average success vs 95.4% full model and 1.718x speedup.
• On SIMPLER with OpenVLA, SAPrune reaches 52.5% average success vs 53.5% full model, while the next best pruned baseline (EfficientVLA) reaches 42.3%.
• On the real AgileX Piper with pi0, SAPrune reduces E2E latency from 88.68 ms to 61.97 ms (11.28 to 16.14 Hz) with 54.0% average success vs 55.5% for pi0.

Surprising findings

• Under a unified 32-token budget, FastV and DivPrune drop success substantially on OpenVLA-OFT (to about 54%), while SAPrune stays near full-model accuracy.
• Calibration transfers across environments: layers calibrated on SIMPLER or real-world rollouts still yield 47.6-51.1% Long success on LIBERO.

Limitations

• A new model requires a separate calibration procedure; transfer to unseen settings must be validated empirically.
• Degradation on the LIBERO Long suite under extreme token reduction.
• Some baselines (FastV, SP-VLA, EfficientVLA) are faster than SAPrune in the unified 32-token setting, since SAPrune processes more tokens in early layers.

Experimental setup

Datasets: LIBERO SIMPLER LIBERO-Plus

Benchmarks: LIBERO SIMPLER LIBERO-Plus real-world AgileX Piper tasks

Metrics: success rate latency speedup FLOPs ratio E2E latency

Baselines: FastV DivPrune SP-VLA VLA-ADP EfficientVLA VLA-Pruner

Builds on: FastV DivPrune EfficientVLA OpenVLA pi0

Closest papers by meaning

• QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.92)
• Hi-Lo Prune: Look at What You'll Lose before Pruning with Hierarchical Token Selection (CVPR 2026 · similarity 0.92)
• TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model (CVPR 2025 · similarity 0.92)
• Each Complexity Deserves a Pruning Policy (NeurIPS 2025 · similarity 0.92)
• ZOO-Prune: Training-Free Token Pruning via Zeroth-Order Gradient Estimation in Vision-Language Models (CVPR 2026 · similarity 0.92)
• LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models (ICLR 2026 · similarity 0.92)