Research map / Vision-language and multimodal / Visual question answering
Playing Lottery Tickets with Vision and Language
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IMP to identify a task-specific sparse mask and reset it to pre-trained UNITER weights | UNITER · VL downstream tasks | accuracy or retrieval R@1: 99% of full accuracy at 50%-70% sparsity | Helped |
| Use a pre-training-derived mask rather than a task-specific mask | UNITER · All downstream tasks | average accuracy: 98%/96% of the full accuracy at 60%/70% sparsity | Helped |
| Transfer a task-specific ticket to other downstream tasks | UNITER · Other VL downstream tasks | task performance | Helped |
| Use random pruning instead of the learned IMP mask (ablation) | UNITER lottery tickets · VL downstream tasks | accuracy or retrieval R@1: 69.98± -> 60.45; 59.26± -> 55.95; 53.15± -> 25.35; 76.32± -> 52.42; 77.69± -> 71.30; 74.06± -> 72.95; 70.15± -> 61.44; 8 | Hurt |
| Replace pre-trained UNITER initialization with pre-trained BERT weights while retaining the learned mask (ablation) | UNITER lottery tickets · VL downstream tasks | accuracy or retrieval R@1: 69.98± -> 67.98; 59.26± -> 58.45; 53.15± -> 50.39; 76.32± -> 54.15; 77.69± -> 76.45; 74.06± -> 71.09; 70.15± -> 63.38; 8 | Hurt |
| Shuffle pre-trained UNITER weights within each layer while retaining the learned mask (ablation) | UNITER lottery tickets · VL downstream tasks | accuracy or retrieval R@1: 69.98± -> 60.46; 59.26± -> 47.49; 53.15± -> 6.25; 76.32± -> 51.52; 77.69± -> 69.32; 74.06± -> 67.34; 70.15± -> 38.94; 83 | Hurt |
| Rewind ticket weights to an early training step instead of resetting to pre-trained weights (ablation) | UNITER lottery tickets · VQA | accuracy | No clear effect |
| Use adversarial training rather than standard cross-entropy training for universal sparse tickets (ablation) | UNITER pre-training-derived tickets · VQA, GQA, VCR, NLVR², SNLI-VE, RefCOCO+ | accuracy: 70.41 -> 70.80; 59.44 -> 59.85; 50.37 -> 51.07; 75.52 -> 76.70; 77.79 -> 77.99; 74.41 -> 74.74 | Helped |
| Use a ticket found through adversarial training at higher sparsity | UNITER · VQA, VCR, RefCOCO+ | performance: 10% sparser to match the performance of a standard trained one | Helped |
| Prune ViLT with IMP | ViLT · VQA and NLVR² | accuracy: 70.51± at 30% sparsity; 75.22± at 30% sparsity | Helped |
| Use random pruning instead of lottery-ticket masks (ablation) | LXMERT · VQA, GQA, NLVR² | accuracy: 69.29± -> 65.22±; 59.40± -> 47.88±; 74.03± -> 51.38± | Hurt |
| Use random pruning instead of lottery-ticket masks (ablation) | ViLT · VQA and NLVR² | accuracy: 70.51± -> 65.16±; 75.22± -> 56.14± | Hurt |
Closest papers by meaning
- The Lottery Ticket Hypothesis for Pre-trained BERT Networks (NeurIPS 2020 · similarity 0.91)
- Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP (ICLR 2020 · similarity 0.91)
- The Lottery Ticket Hypothesis for Object Recognition (CVPR 2021 · similarity 0.90)
- Data-Efficient Double-Win Lottery Tickets from Robust Pre-training (ICML 2022 · similarity 0.90)
- The Lottery Tickets Hypothesis for Supervised and Self-Supervised Pre-Training in Computer Vision Models (CVPR 2021 · similarity 0.90)
- Weisfeiler and Leman Go Gambling: Why Expressive Lottery Tickets Win (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.