Research map / Vision-language and multimodal / Human and embodied modeling
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace region or deep convolutional visual embedding with linear patch projection | ViLT-B/32 · VQAv2 test-dev | VQA score: 75.95 -> 70.33 | Hurt |
| replace region or deep convolutional visual embedding with linear patch projection | ViLT-B/32 · inference latency | time (ms): ~900 -> ~15 | Helped |
| add whole word masking during pre-training (ablation) | ViLT-B/32 · VQAv2 test-dev | VQA score: 70.16 ± 0.01 -> 70.33 ± 0.01 | Helped |
| add masked patch prediction during pre-training (ablation) | ViLT-B/32 · VQAv2 test-dev | VQA score: 70.33 ± 0.01 -> 70.21 ± 0.05 | Hurt |
| add RandAugment during fine-tuning (ablation) | ViLT-B/32 · VQAv2 test-dev | VQA score: 70.33 ± 0.01 -> 70.85 ± 0.13 | Helped |
| increase pre-training from 25K to 100K steps (ablation) | ViLT-B/32 · VQAv2 test-dev | VQA score: 68.96 ± 0.07 -> 70.16 ± 0.01 | Helped |
| increase pre-training from 100K to 200K steps (ablation) | ViLT-B/32 · Flickr30k (1K) text retrieval | R@1: 83.69 (69.73) -> 83.50 (73.24) | Hurt |
Closest papers by meaning
- Probing Visual Language Priors in VLMs (ICML 2025 · similarity 0.90)
- Unveiling Encoder-Free Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Towards Better Vision-Inspired Vision-Language Models (CVPR 2024 · similarity 0.89)
- Locality Alignment Improves Vision-Language Models (ICLR 2025 · similarity 0.88)
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Frozen Transformers in Language Models Are Effective Visual Encoder Layers (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.