Research map / Vision-language and multimodal / Human and embodied modeling
NewsStories: Illustrating Articles with Visual Summaries
Reuben Tan, Bryan A. Plummer, Kate Saenko, JP Lewis, Avneesh Sud, Thomas Leung
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune CLIP on NewsStories with single image-text InfoNCE | CLIP · GoodNews zero-shot (5 images) | R@1: 20.90 -> 15.61 | Hurt |
| fine-tune CLIP on NewsStories with MIL-NCE | CLIP · GoodNews zero-shot (5 images) | R@1: 20.90 -> 13.30 | Hurt |
| aggregate multiple images (mean) instead of single-image alignment | CLIP fine-tuning · NewsStories test (5 images) | R@1: 35.88 -> 49.12 | Improved |
| replace mean aggregation with an image transformer | Mean baseline · NewsStories test (5 images) | R@1: 49.12 -> 50.08 | Improved |
| add sentence-to-image MIL alignment (MIL-SIM) on top of mean aggregation (ablation) | Mean · NewsStories test (5 images) | R@1: 49.12 -> 54.24 | Improved |
| add MIL-SIM objective (ablation) | Mean · GoodNews zero-shot (5 images per set) | R@1: 28.77 -> 30.23 | Improved |
| add MIL-SIM objective (ablation) | Mean · GoodNews zero-shot (variable set size) | R@1: 21.30 -> 25.12 | Improved |
| train on NewsStories with MIL-SIM versus pretrained CLIP | CLIP · NewsStories test (5 images) | R@1: 31.03 -> 54.24 | Improved |
| increase MIL-SIM input text length from 32 to 256 tokens (ablation) | MIL-SIM · NewsStories val (5 images) | R@1: 38.08 -> 49.23 | Improved |
| increase MIL-SIM input text length from 256 to 512 tokens (ablation) | MIL-SIM · NewsStories val (5 images) | R@1: 49.23 -> 48.73 | Hurt |
| increase MIL-SIM input text length from 256 to 512 tokens (ablation) | MIL-SIM · GoodNews zero-shot (5 images) | R@1: 30.23 -> 27.06 | Hurt |
| use single-image one-to-one alignment as the number of images per set grows | Single Image · GoodNews zero-shot | R@1: 17.27 -> 15.61 | Hurt |
Closest papers by meaning
- A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (CVPR 2024 · similarity 0.87)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.86)
- Semantic and Expressive Variations in Image Captions Across Languages (CVPR 2025 · similarity 0.86)
- FG-CLIP: Fine-Grained Visual and Textual Alignment (ICML 2025 · similarity 0.86)
- LoTLIP: Improving Language-Image Pre-training for Long Text Understanding (NeurIPS 2024 · similarity 0.85)
- CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
