Research map / Video / Video-language understanding
Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanism
Tao Chen, Kun Zhang, Qiong Wu, Xiao Chen, Chao Chang, Xiaoshuai Sun, Yiyi Zhou, Rongrong Ji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add FlexMem visual memory | LLaVA-Video 7B · TimeScope | accuracy: 65.0 -> 85.9 | Improved |
| add FlexMem visual memory | LLaVA-Video 7B · LVBench | accuracy: 42.6 -> 51.0 | Improved |
| add FlexMem visual memory | LLaVA-OneVision 7B · TimeScope | accuracy: 56.3 -> 80.5 | Improved |
| add FlexMem versus AKS retrieval on Video-MME | LLaVA-Video 7B · Video-MME (all) | accuracy: 65.3 -> 64.7 | Hurt |
| add FlexMem versus AdaReTaKe under 24GB limit | LLaVA-Video 7B · LVBench (val), 3090 | accuracy: 46.8 -> 50.2 | Improved |
| combine context and local compression (dual-pathway) vs context only (ablation) | FlexMem · LongVideoBench (all) | accuracy: 62.5 -> 63.6 | Improved |
| combine context and local compression (dual-pathway) vs local only (ablation) | FlexMem · LongVideoBench (all) | accuracy: 62.6 -> 63.6 | Improved |
| use both context and local memory during prefill (ablation) | FlexMem · LongVideoBench (all) | accuracy: 61.9 -> 63.6 | Improved |
| recall only relevant memories at decoding instead of all memory bank (ablation) | FlexMem · LongVideoBench (all) | accuracy: 59.8 -> 63.6 | Improved |
| use smaller clips (8 frames) instead of 32 (ablation) | FlexMem · LongVideoBench (all) | accuracy: 61.7 -> 63.6 | Improved |
| replace encoding-based index with fast MemIndex (ablation) | FlexMem · MLVU M-avg | accuracy: 72.4 -> 72.3 | No clear effect |
| replace encoding-based index with fast MemIndex (last-token, 3 layers) (ablation) | FlexMem · LVBench (val) | accuracy: 51.0 -> 46.8 | Hurt |
Closest papers by meaning
- MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding (CVPR 2024 · similarity 0.91)
- FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding (NeurIPS 2026 · similarity 0.90)
- Streaming Long Video Understanding with Large Language Models (NeurIPS 2024 · similarity 0.90)
- MovieChat: From Dense Token to Sparse Memory for Long Video Understanding (CVPR 2024 · similarity 0.89)
- ReWind: Understanding Long Videos with Instructed Learnable Memory (CVPR 2025 · similarity 0.89)
- Leveraging Textual Memory and Key Frame Reasoning for Full Video Understanding Using Off-the-Shelf LLMs and VLMs (Student Abstract) (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
