ICCV 2023 accepted papers on Vision-language and multimodal
161 of the 2,156 papers accepted at ICCV 2023 (7.5%) are on Vision-language and multimodal, compared with 4.5% at ICCV 2021. They fall into 6 topics.
See ICCV 2023 in the interactive map
Submitting to ICCV? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
CLIP and zero-shot · 66 papers
- Sigmoid Loss for Language Image Pre-Training (1,889 citations)
- Prompt-aligned Gradient for Prompt Tuning (393 citations)
- Self-regulating Prompts: Foundational Model Adaptation without Forgetting (314 citations)
- What Does a Platypus Look Like? Generating Customized Prompts for Zero-Shot Image Classification (297 citations)
- Zero-Shot Composed Image Retrieval with Textual Inversion (180 citations)
- MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training for X-ray Diagnosis (156 citations)
- Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models (122 citations)
- SuS-X: Training-Free Name-Only Transfer of Vision-Language Models (122 citations)
- Teaching CLIP to Count to Ten (121 citations)
- Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning (118 citations)
- Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement (111 citations)
- Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation (99 citations)
and 54 more in the interactive map.
Human and embodied modeling · 37 papers
- SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding (231 citations)
- What does CLIP know about a red circle? Visual prompt engineering for VLMs (194 citations)
- 3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment (162 citations)
- EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes (95 citations)
- Multi3DRefer: Grounding Text Description to Multiple 3D Objects (94 citations)
- Revisiting Scene Text Recognition: A Data Perspective (87 citations)
- Vision Grid Transformer for Document Layout Analysis (80 citations)
- I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervision (48 citations)
- Dynamic Perceiver for Efficient Visual Recognition (43 citations)
- Learning Concise and Descriptive Attributes for Visual Recognition (43 citations)
- Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World (43 citations)
- Gender Artifacts in Visual Datasets (38 citations)
and 25 more in the interactive map.
Cross-modal learning · 27 papers
- Cross-Modal Translation and Alignment for Survival Analysis (146 citations)
- Multi-Modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion (127 citations)
- Distribution-Consistent Modal Recovering for Incomplete Multimodal Learning (119 citations)
- Boosting Multi-modal Model Performance with Adaptive Gradient Modulation (81 citations)
- Multimodal Variational Auto-encoder based Audio-Visual Segmentation (62 citations)
- Cross-Modal Orthogonal High-Rank Augmentation for RGB-Event Transformer-Trackers (53 citations)
- Audio-Visual Class-Incremental Learning (52 citations)
- Unify, Align and Refine: Multi-Level Semantic Alignment for Radiology Report Generation (49 citations)
- Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learning (48 citations)
- Towards Unifying Medical Vision-and-Language Pre-Training via Soft Prompts (45 citations)
- AutoAD II: The Sequel - Who, When, and What in Movie Audio Description (42 citations)
- Scratch Each Other's Back: Incomplete Multi-Modal Brain Tumor Segmentation via Category Aware Group Self-Support Learning (36 citations)
and 15 more in the interactive map.
Robot action and navigation · 11 papers
- AerialVLN: Vision-and-Language Navigation for UAVs (126 citations)
- GridMM: Grid Memory Map for Vision-and-Language Navigation (123 citations)
- Scaling Data Generation in Vision-and-Language Navigation (98 citations)
- Bird's-Eye-View Scene Graph for Vision-Language Navigation (83 citations)
- DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation (67 citations)
- ARNOLD: A Benchmark for Language-Grounded Task Learning with Continuous States in Realistic 3D Scenes (53 citations)
- Learning Navigational Visual Representations with Semantic Map Supervision (51 citations)
- Learning Vision-and-Language Navigation from YouTube Videos (40 citations)
- Skill Transformer: A Monolithic Policy for Mobile Manipulation (25 citations)
- Pretrained Language Models as Visual Planners for Human Assistance (23 citations)
- Manipulate by Seeing: Creating Manipulation Controllers from Pre-Trained Representations (13 citations)
Multimodal LLMs · 10 papers
- ViperGPT: Visual Inference via Python Execution for Reasoning (490 citations)
- Document Understanding Dataset and Evaluation (DUDE) (95 citations)
- All in Tokens: Unifying Output Space of Visual Tasks via Soft Token (65 citations)
- Encyclopedic VQA: Visual Questions About Detailed Properties of Fine-Grained Categories (61 citations)
- UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding (57 citations)
- Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images (54 citations)
- LogicSeg: Parsing Visual Semantics with Neural Logic Learning and Reasoning (49 citations)
- Improving Unsupervised Visual Program Inference with Code Rewriting Families (8 citations)
- SINC: Self-Supervised In-Context Learning for Vision-Language Tasks (7 citations)
- VLSlice: Interactive Vision-and-Language Slice Discovery (7 citations)
Visual question answering · 10 papers
- TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering (270 citations)
- PromptCap: Prompt-Guided Image Captioning for VQA with GPT-3 (203 citations)
- VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answering (88 citations)
- Variational Causal Inference Network for Explanatory Visual Question Answering (27 citations)
- Decouple Before Interact: Multi-Modal Prompt Learning for Continual Visual Question Answering (24 citations)
- Towards Models that Can See and Read (15 citations)
- VQA Therapy: Exploring Answer Differences by Visually Grounding Answers (14 citations)
- Sentence Attention Blocks for Answer Grounding (7 citations)
- Toward Multi-Granularity Decision-Making: Explicit Visual Reasoning with Hierarchical Knowledge (4 citations)
- Toward Unsupervised Realistic Visual Question Answering (4 citations)
