Research map / Video / Video-language understanding
InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InfiniPot-V, a training-free, query-agnostic continual KV cache compression framework that caps memory for streaming video MLLMs using TaR (temporal key redundancy) and VaN (value-norm saliency) token selection.
Problem
KV cache of multimodal LLMs grows linearly with video stream length, exceeding fixed memory on edge devices; existing compression methods need the full cache or the user query before compressing.
Approach
Continually compress the KV cache whenever a fixed memory threshold is hit, selecting tokens without queries by combining temporal key-redundancy scoring (TaR, patch-wise cosine similarity of keys across frames) with value-norm semantic scoring (VaN, with layer-adaptive pooling based on coefficient of variation).
Key results
- Under a 6K memory budget, Qwen-2-VL-7B with InfiniPot-V reaches 64.3 average on MLVU/VideoMME-style breakdown versus 64.2 for full KV at 50K tokens, roughly 88% cache reduction.
- On Jetson AGX Orin with Qwen-2.5-VL-3B, peak memory stays at 9.2-10.7 GB while Full KV grows from 13.8 to 39.0 GB over 100-500 s streams, a 3.6x reduction at 500 s; Full KV OOMs at 600 s.
- On OVO-Bench with Qwen-2.5-VL-7B under 4K budget, InfiniPot-V improves OVO-Avg from 51.7 (uniform select) to 53.6 and StreamingBench from 75.2 to 76.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply InfiniPot-V with 6K budget instead of full KV | Qwen-2-VL-7B · EgoSchema | accuracy: 65.2 -> 65.6 | Helped |
| Apply InfiniPot-V with 6K budget instead of full KV | Qwen-2-VL-7B · MLVU | accuracy: 65.8 -> 65.8 | No clear effect |
| Apply InfiniPot-V with 6K budget instead of full KV | Qwen-2-VL-7B · VideoMME | accuracy: 63.9 -> 62.8 | Hurt |
| Apply InfiniPot-V with 6K budget instead of full KV | LLaVA-Next-7B · LongVideoBench | accuracy: 63.5 -> 60.9 | Hurt |
| Replace uniform selection with InfiniPot-V compression | Qwen-2.5-VL-7B · OVO-Bench | average accuracy: 51.7 -> 53.6 | Helped |
| Use reversed TaR selection rather than retaining distinctive tokens (ablation) | InfiniPot-V on Qwen-2-VL-7B · MLVU dev | average accuracy: 64.5 -> 55.5 | Hurt |
| Use frame-level rather than patch-wise TaR similarity (ablation) | InfiniPot-V on Qwen-2-VL-7B · MLVU dev | average accuracy: 64.5 -> 62.9 | Hurt |
| Use reversed VaN selection rather than high-VaN tokens (ablation) | InfiniPot-V on Qwen-2-VL-7B · MLVU dev | average accuracy: 63.0 -> 55.0 | Hurt |
| Add adaptive pooling to VaN (ablation) | InfiniPot-V on Qwen-2-VL-7B · MLVU dev | average accuracy: 63.0 -> 64.1 | Helped |
| Combine TaR and VaN with pooling rather than using either alone (ablation) | InfiniPot-V on Qwen-2-VL-7B · MLVU dev | average accuracy: 64.1 -> 65.8 | Helped |
| Set TaR allocation to α=0.6 rather than VaN-only α=0 (ablation) | InfiniPot-V on Qwen-2-VL-7B · VideoMME | average accuracy: 62.1 -> 63.1 | Helped |
| Increase recent-frame proportion from r=f×0.125 to r=f×0.5 (ablation) | TaR-only compression on Qwen-2-VL-7B · MLVU | average accuracy: 63.1 -> 61.3 | Hurt |
Surprising findings
- Query-guided compression (SnapKV) built on a single query degrades on follow-up queries in multi-turn dialogue, while query-agnostic InfiniPot-V stays accurate.
- Attention scores from generic or last-vision-token queries drop accuracy substantially (60.32 vs 68.75 full KV) even without a memory constraint.
Limitations
- Positional indices are re-assigned for streaming, discarding original positional information, especially problematic for Qwen-2-VL 3D RoPE.
- Hyperparameters (alpha, r, compression ratio, CV thresholds) are tuned per model and require pre-computed model-specific CV thresholds.
Experimental setup
Datasets: VideoMME MLVU LongVideoBench EgoSchema RVS-Ego RVS-Movie OVO-Bench StreamingBench
Benchmarks: VideoMME MLVU LongVideoBench EgoSchema RVS-Ego RVS-Movie OVO-Bench StreamingBench
Metrics: accuracy score peak GPU memory FPS throughput power
Baselines: Uniform Select SnapKV InfiniPot ReKV ReKV w/o offloading LongVU STC DyCoke TTM FastV GPT-4o LLaVA-OV
Builds on: SnapKV ReKV LongVU DyCoke InfiniPot
Closest papers by meaning
- CoRDS: Coreset-Based Representative and Diverse Selection for Streaming Video Understanding (NeurIPS 2026 · similarity 0.89)
- VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion (NeurIPS 2026 · similarity 0.88)
- EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction (NeurIPS 2026 · similarity 0.88)
- VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration (ICLR 2025 · similarity 0.88)
- One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding (NeurIPS 2025 · similarity 0.88)
- DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.