Research map / Video / Video-language understanding
Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding
Christopher Clark, Jieyu Zhang, Zixian Ma, Jae Sung Park, Rohun Tripathi, Sangho Lee, Mohammadreza Salehi, Jason Ren, Chris Dongjoo Kim, Yinuo Yang, Vincent Shao, Yue Yang and 8 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove bidirectional attention between vision tokens (ablation) | Molmo2-4B (video-only) · video benchmarks | Caption F1: 32.6 -> 30.8 | Hurt |
| remove bidirectional attention between vision tokens (ablation) | Molmo2-4B (video-only) · video QA | QA avg.: 64.8 vs 64.4 without | Improved |
| add token weighting (ablation) | Molmo2-4B (video-only) · video QA | QA avg.: 64.0 -> 64.8 | Improved |
| add token weighting (ablation) | Molmo2-4B (video-only) · video captioning | Caption F1: 34.0 -> 32.6 | Hurt |
| add Molmo2-Cap and Molmo2-QA to academic video SFT data (ablation) | Molmo2-4B (video-only) · video SFT | Caption F1: 4.7 -> 32.6 | Improved |
| use merged video+frame captions instead of human transcript captions only (ablation) | Molmo2-4B captioner · Molmo2-Cap | Caption F1: 22.1 -> 33.2 | Improved |
| add video-only or frame-only captions to merged captions (ablation) | Molmo2-4B captioner · Molmo2-Cap | Caption F1: 33.2 -> 32.3 | No clear effect |
| train jointly with images instead of video-only (ablation) | Molmo2-4B · video benchmarks | QA avg.: 64.8 -> 66.7 | Improved |
| predict points before counting (ablation) | Molmo2-4B · Molmo2-VideoCount | accuracy: 28.1 -> 34.5 | Improved |
| upsample medium and high-count pointing examples (ablation) | Molmo2-4B · Molmo2-VideoPoint | F1: 28.1 -> 31.8 | Improved |
| train specialized pointing-only model instead of joint model (ablation) | Molmo2-4B · Molmo2-VideoPoint | F1: 28.4 -> 31.8 | Improved |
| add point-based single-object tracking sub-task (ablation) | Molmo2-4B · Academic VOS tracking | HOTA: 66.2 -> 66.7 vs 67.2 with grounding | Hurt |
Closest papers by meaning
- Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models (CVPR 2025 · similarity 0.92)
- MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models (CVPR 2025 · similarity 0.90)
- ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models (ICLR 2024 · similarity 0.89)
- UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling (NeurIPS 2024 · similarity 0.89)
- PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding (NeurIPS 2025 · similarity 0.88)
- CogVLM: Visual Expert for Pretrained Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
