AI Paper Atlas

Research map / Video / Video-language understanding

MSJoE: Jointly Evolving MLLM and Sampler for Efficient Long-Form Video Understanding

Wenhui Tan, Xiaoyi Yu, Jiaze Li, Yijing Chen, Jianzhong Ju, Zhenbo Luo, Ruihua Song, Jian Luan

CVPR 2026 · Video › Video-language understanding · approach: Long video understanding · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
jointly train MLLM and sampler instead of freezing the MLLM (ablation)MSJoE · VideoMME-Long (32 frames)accuracy: 52.2 -> 54.1Improved
train the MLLM to generate queries (vs frozen MLLM) with same sampled frames (ablation)MSJoE · LVBench (32 frames)accuracy: 44.8 -> 46.4Improved
add sampler pre-training before joint RL (ablation)MSJoE · MLVU, LoVi, VLong, LVB (32 frames)accuracy: MLVU 69.1 -> 69.3, LVBench 46.2 -> 46.4Improved
train the sampler during RL instead of keeping it frozen (MLLM trained) (ablation)MSJoE · LongVideoBench (32 frames)accuracy: 58.1 -> 60.1Improved
use the question as CLIP query with top-k instead of uniform sampling (ablation)Qwen2.5-VL-7B frozen · MLVU, VideoMME-Long, LVBench (32 frames)accuracy: MLVU 61.5 -> 68.3, VLong 49.9 -> 48.9 (lower)Improved
decompose question into multiple queries with top-k pooling (no trained sampler) (ablation)frozen Qwen2.5-VL-7B · MLVU (32 frames)accuracy: 68.3 -> 64.5Hurt
replace top-k with a trained sampler for multi-query similarity (frozen MLLM) (ablation)frozen Qwen2.5-VL-7B · VideoMME-Long (32 frames)accuracy: 45.4 -> 52.2Improved
naive top-k similarity sampling vs uniform sampling at varying frame budgets (ablation)Qwen2.5-VL-7B · LongVideoBench and VideoMME-LongaccuracyHurt
apply MSJoE instead of uniform samplingQwen2.5-VL-7B (64 frames) · LVBenchaccuracy: 39.2 -> 51.1Improved
apply MSJoE instead of TSPO (learned sampler with frozen MLLM)Qwen2.5-VL-7B (64 frames) · MLVU, VideoMME-Long, LVBenchaccuracy: MLVU 74.3 -> 75.1, LoVi 64.2 -> 62.2Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy