CVPR 2022 accepted papers on Vision-language and multimodal
119 of the 2,077 papers accepted at CVPR 2022 (5.7%) are on Vision-language and multimodal, compared with 5.0% at CVPR 2021. They fall into 5 topics.
See CVPR 2022 in the interactive map
Submitting to CVPR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Human and embodied modeling · 48 papers
- LAVT: Language-Aware Vision Transformer for Referring Image Segmentation (397 citations)
- An Empirical Study of Training End-to-End Vision-and-Language Transformers (338 citations)
- Rethinking Visual Geo-Localization for Large-Scale Applications (263 citations)
- Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality (214 citations)
- Negative-Aware Attention Framework for Image-Text Matching (204 citations)
- VisualGPT: Data-Efficient Adaptation of Pretrained Language Models for Image Captioning (184 citations)
- Improving Visual Grounding With Visual-Linguistic Verification and Iterative Reasoning (167 citations)
- MERLOT Reserve: Neural Script Knowledge Through Vision and Language and Sound (157 citations)
- Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation (134 citations)
- Injecting Semantic Concepts Into End-to-End Image Captioning (131 citations)
- Comprehending and Ordering Semantics for Image Captioning (128 citations)
- Shifting More Attention to Visual Backbone: Query-Modulated Refinement Networks for End-to-End Visual Grounding (103 citations)
and 36 more in the interactive map.
Cross-modal learning · 29 papers
- FLAVA: A Foundational Language and Vision Alignment Model (532 citations)
- Balanced Multimodal Learning via On-the-Fly Gradient Modulation (336 citations)
- Vision-Language Pre-Training With Triple Contrastive Learning (290 citations)
- Multimodal Token Fusion for Vision Transformers (246 citations)
- Omnivore: A Single Model for Many Visual Modalities (196 citations)
- Multimodal Dynamics: Dynamical Fusion for Trustworthy Multimodal Classification (187 citations)
- Are Multimodal Transformers Robust to Missing Modality? (183 citations)
- Everything at Once - Multi-Modal Fusion Transformer for Video Retrieval (143 citations)
- Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation (128 citations)
- Expressive Talking Head Generation With Granular Audio-Visual Control (119 citations)
- Cross Modal Retrieval With Querybank Normalisation (99 citations)
- ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval (86 citations)
and 17 more in the interactive map.
CLIP and zero-shot · 20 papers
- Conditional Prompt Learning for Vision-Language Models (1,735 citations)
- CRIS: CLIP-Driven Referring Image Segmentation (401 citations)
- LiT: Zero-Shot Transfer With Locked-Image Text Tuning (380 citations)
- VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks (310 citations)
- Decoupling Zero-Shot Semantic Segmentation (292 citations)
- Prompt Distribution Learning (245 citations)
- Unified Contrastive Learning in Image-Text-Label Space (193 citations)
- General Facial Representation Learning in a Visual-Linguistic Manner (178 citations)
- Align and Prompt: Video-and-Language Pre-Training With Entity Prompts (175 citations)
- ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic (142 citations)
- CLIP-Event: Connecting Text and Images With Event Structures (106 citations)
- Retrieval Augmented Classification for Long-Tail Visual Recognition (77 citations)
and 8 more in the interactive map.
Visual question answering · 13 papers
- MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-Based Visual Question Answering (128 citations)
- Invariant Grounding for Video Question Answering (118 citations)
- Learning To Answer Questions in Dynamic Audio-Visual Scenarios (114 citations)
- Transform-Retrieve-Generate: Natural Language-Centric Outside-Knowledge Visual Question Answering (91 citations)
- LaTr: Layout-Aware Transformer for Scene-Text VQA (89 citations)
- WebQA: Multihop and Multimodal QA (69 citations)
- Grounding Answers for Visual Questions Asked by Visually Impaired People (50 citations)
- SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering (43 citations)
- Dual-Key Multimodal Backdoors for Visual Question Answering (35 citations)
- Maintaining Reasoning Consistency in Compositional Visual Question Answering (29 citations)
- V-Doc: Visual Questions Answers With Documents (21 citations)
- Query and Attention Augmentation for Knowledge-Based Explainable Reasoning (15 citations)
and 1 more in the interactive map.
Robot action and navigation · 9 papers
- Think Global, Act Local: Dual-Scale Graph Transformer for Vision-and-Language Navigation (176 citations)
- Simple but Effective: CLIP Embeddings for Embodied AI (164 citations)
- Cross-Modal Map Learning for Vision and Language Navigation (78 citations)
- Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation (55 citations)
- Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigation (52 citations)
- ADAPT: Vision-Language Navigation With Modality-Aligned Action Prompts (48 citations)
- Less Is More: Generating Grounded Navigation Instructions From Landmarks (44 citations)
- Reinforced Structured State-Evolution for Vision-Language Navigation (42 citations)
- HOP: History-and-Order Aware Pre-Training for Vision-and-Language Navigation (8 citations)
