Research map / Video / Video-language understanding

V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models

NeurIPS 2026 · Video › Video-language understanding · approach: Long video understanding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes V-CAST, a training-free, plug-and-play token pruning method for VideoLLMs that allocates per-frame budgets via semantic-trajectory curvature and selects tokens with a dual-anchor diversity-saliency score while keeping on-grid coordinates for MRoPE alignment.

Problem

Long-context video inference in VideoLLMs is dominated by redundant visual tokens in prefill; under tight budgets, existing methods give discontinuous spatio-temporal coverage (coarse or segment-based budgeting) and token merging misaligns positions under MRoPE.

Approach

Treat per-frame mean token embeddings as a trajectory on the hypersphere, compute turning-angle curvature, and route budgets via a softmax (entropy-regularized) over curvature; within frames, select top tokens by a sum of contextual diversity (1 - cos to frame mean) and L2-norm activation, retaining original coordinates instead of merging.

Key results

• On Qwen3-VL-8B with 32 frames at 25% retention, V-CAST scores 63.3 average (98.6% of vanilla) vs 62.6 for second-best VidCom^2 (96.6%).
• On LLaVA-OV-7B at 25% retention, V-CAST averages 58.8 (99.5% of original), best among compared methods.
• Under 64-frame Qwen3-VL-8B input at 25% retention, V-CAST reduces total latency by 13.6% and peak memory by 13.0% relative to vanilla in the efficiency table (32-frame VideoMME setting).

Surprising findings

• Token merging (HoliTom) underperforms a naive AvgPool baseline on MVBench, LongVideoBench and VideoMME, suggesting merging misaligns MRoPE positions.
• FlashVID runs out of memory under 64-frame Qwen3-VL inputs because attention-guided compression inside the LLM adds overhead.

Limitations

• Efficiency gains are measured in a single experimental setup and reported as wall-clock time that depends on hardware.
• Temperature and anchor weights are set by default values with sensitivity checks only on one model.

Experimental setup

Datasets: MVBench LongVideoBench MLVU VideoMME EgoSchema

Benchmarks: MVBench LongVideoBench MLVU VideoMME EgoSchema

Metrics: accuracy average score percent of original performance peak GPU memory prefill latency total latency throughput

Closest papers by meaning

• FastVID: Dynamic Density Pruning for Fast Video Large Language Models (NeurIPS 2025 · similarity 0.92)
• HoliTom: Holistic Token Merging for Fast Video Large Language Models (NeurIPS 2025 · similarity 0.91)
• Sharp Eyes and Memory for VideoLLMs: Information-Aware Visual Token Pruning for Efficient and Reliable VideoLLM Reasoning (AAAI 2026 · similarity 0.90)
• ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning (AAAI 2026 · similarity 0.89)
• DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models (CVPR 2025 · similarity 0.89)
• Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization (ICML 2024 · similarity 0.89)