Research map / Video / Video-language understanding
Less is More: Picking Informative Frames for Video Captioning
Yangyu Chen, Shuhui Wang, Weigang Zhang, Qingming Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add language reward to visual-diversity-only PickNet (ablation) | PickNet · MSVD test | CIDEr: 75.6 -> 76.0 | Improved |
| add visual diversity reward to language-only PickNet (ablation) | PickNet · MSVD test | CIDEr: 74.7 -> 76.0 | Improved |
| add language reward to visual-diversity-only PickNet (ablation) | PickNet · MSVD test | BLEU@4: 43.6 -> 46.1 | Improved |
| add visual diversity reward to language-only PickNet (ablation) | PickNet · MSVD test | BLEU@4: 49.9 -> 46.1 | Hurt |
| replace all-30-frame encoding with learned frame picking (about 6 frames) | Encoder-Decoder baseline · MSVD test | CIDEr: 69.4 -> 76.0 | Improved |
| replace random frame selection with PickNet picking | Encoder-Decoder · MSVD test | CIDEr: 49.2 -> 76.0 | Improved |
| replace k-means frame selection with PickNet picking | Encoder-Decoder · MSVD test | CIDEr: 70.9 -> 76.0 | Improved |
| replace all-30-frame encoding with PickNet picking (about 6 frames) | Encoder-Decoder baseline · MSVD test | relative running time: 5x -> 1x | Improved |
| replace all-30-frame encoding with PickNet picking (about 8 frames) | Encoder-Decoder baseline · MSR-VTT test | CIDEr: 41.2 -> 42.1 | Improved |
| add language reward to visual-diversity-only PickNet (ablation) | PickNet · MSR-VTT test | CIDEr: 41.0 -> 42.1 | Improved |
| add visual diversity reward to language-only PickNet (ablation) | PickNet · MSR-VTT test | CIDEr: 41.9 -> 42.1 | Improved |
| replace random frame selection with PickNet picking | Encoder-Decoder · MSR-VTT test | CIDEr: 32.6 -> 42.1 | Improved |
Closest papers by meaning
- Progress-Aware Video Frame Captioning (CVPR 2025 · similarity 0.88)
- Efficient Frame Selection for Long Video Understanding via Reinforcement Learning (CVPR 2026 · similarity 0.88)
- Video Captioning via Hierarchical Reinforcement Learning (CVPR 2018 · similarity 0.87)
- An Efficient Framework for Dense Video Captioning (AAAI 2020 · similarity 0.87)
- MAMS: Model-Agnostic Module Selection Framework for Video Captioning (AAAI 2025 · similarity 0.87)
- Fine-Grained Captioning of Long Videos through Scene Graph Consolidation (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
