CVPR 2023 accepted papers on Vision-language and multimodal

177 of the 2,359 papers accepted at CVPR 2023 (7.5%) are on Vision-language and multimodal, compared with 5.7% at CVPR 2022. They fall into 5 topics.

See CVPR 2023 in the interactive map

Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.

Topics and papers

Human and embodied modeling · 69 papers

Image as a Foreign Language: BEiT Pretraining for Vision and Vision-Language Tasks (531 citations)
Autoregressive Visual Tracking (351 citations)
Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective (315 citations)
Explicit Visual Prompting for Low-Level Structure Segmentations (210 citations)
Learning To Exploit Temporal Structure for Biomedical Vision-Language Processing (205 citations)
Interactive and Explainable Region-Guided Radiology Report Generation (201 citations)
Images Speak in Images: A Generalist Painter for In-Context Visual Learning (192 citations)
METransformer: Radiology Report Generation by Transformer With Multiple Learnable Expert Tokens (172 citations)
Joint Visual Grounding and Tracking With Natural Language Specification (139 citations)
KiUT: Knowledge-Injected U-Transformer for Radiology Report Generation (132 citations)
Fine-Grained Image-Text Matching by Cross-Modal Hard Aligning Network (122 citations)
Unifying Vision, Text, and Layout for Universal Document Processing (119 citations)

and 57 more in the interactive map.

CLIP and zero-shot · 48 papers

MaPLe: Multi-Modal Prompt Learning (886 citations)
Reproducible Scaling Laws for Contrastive Language-Image Learning (626 citations)
WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation (526 citations)
Visual Prompt Multi-Modal Tracking (347 citations)
Visual-Language Prompt Tuning With Knowledge-Guided Context Optimization (244 citations)
ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding (241 citations)
Scaling Language-Image Pre-Training via Masking (236 citations)
Generalized Decoding for Pixel, Image, and Language (187 citations)
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining (169 citations)
Prompt, Generate, Then Cache: Cascade of Foundation Models Makes Strong Few-Shot Learners (162 citations)
CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not (138 citations)
SmallCap: Lightweight Image Captioning Prompted With Retrieval Augmentation (128 citations)

and 36 more in the interactive map.

Cross-modal learning · 41 papers

ImageBind: One Embedding Space To Bind Them All (810 citations)
Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval (320 citations)
Decoupled Multimodal Distilling for Emotion Recognition (220 citations)
Multi-Modal Learning With Missing Modality via Shared-Specific Feature Modelling (210 citations)
Delivering Arbitrary-Modal Semantic Segmentation (185 citations)
Bridging Search Region Interaction With Template for RGB-T Tracking (147 citations)
Multimodal Prompting With Missing Modalities for Visual Recognition (138 citations)
PMR: Prototypical Modal Rebalance for Multimodal Learning (136 citations)
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning With Multimodal Models (126 citations)
Learning Semantic Relationship Among Instances for Image-Text Matching (115 citations)
Efficient RGB-T Tracking via Cross-Modality Distillation (110 citations)
Align and Attend: Multimodal Summarization With Dual Contrastive Losses (100 citations)

and 29 more in the interactive map.

Visual question answering · 14 papers

Prompting Large Language Models With Answer Heuristics for Knowledge-Based Visual Question Answering (212 citations)
Visual Programming: Compositional Visual Reasoning Without Training (202 citations)
From Images to Textual Prompts: Zero-Shot Visual Question Answering With Frozen Large Language Models (138 citations)
REVEAL: Retrieval-Augmented Visual-Language Pre-Training With Multi-Source Multimodal Knowledge Memory (85 citations)
Generative Bias for Robust Visual Question Answering (64 citations)
Discovering the Real Association: Multimodal Causal Reasoning in Video Question Answering (47 citations)
RMLVQA: A Margin Loss Approach for Visual Question Answering With Language Biases (42 citations)
Are Deep Neural Networks SMARTer Than Second Graders? (26 citations)
MixPHM: Redundancy-Aware Parameter-Efficient Tuning for Low-Resource Visual Question Answering (20 citations)
Super-CLEVR: A Virtual Benchmark To Diagnose Domain Robustness in Visual Reasoning (19 citations)
Q: How To Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images! (12 citations)
Improving Selective Visual Question Answering by Learning From Your Peers (10 citations)

and 2 more in the interactive map.