ICCV 2021 accepted papers on Vision-language and multimodal
72 of the 1,612 papers accepted at ICCV 2021 (4.5%) are on Vision-language and multimodal, compared with 5.4% at ICCV 2019. They fall into 6 topics.
See ICCV 2021 in the interactive map
Submitting to ICCV? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Human and embodied modeling · 24 papers
- MDETR - Modulated Detection for End-to-End Multi-Modal Understanding (809 citations)
- TransVG: End-to-End Visual Grounding With Transformers (428 citations)
- DocFormer: End-to-End Transformer for Document Understanding (410 citations)
- Vision-Language Transformer and Query Generation for Referring Segmentation (303 citations)
- Image Retrieval on Real-Life Images With Pre-Trained Vision-and-Language Models (273 citations)
- From Two to One: A New Scene Text Recognizer With Visual Language Modeling Network (193 citations)
- Describing and Localizing Multiple Changes With Transformers (127 citations)
- Understanding and Evaluating Racial Biases in Image Captioning (97 citations)
- E-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks (79 citations)
- MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning (72 citations)
- Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition (69 citations)
- Learning Attribute-Driven Disentangled Representations for Interactive Fashion Retrieval (57 citations)
and 12 more in the interactive map.
Cross-modal learning · 22 papers
- Joint Audio-Visual Deepfake Detection (240 citations)
- RFNet: Region-Aware Fusion Network for Incomplete Multi-Modal Brain Tumor Segmentation (211 citations)
- CrossCLR: Cross-Modal Contrastive Learning for Multi-Modal Video Representations (153 citations)
- LapsCore: Language-Guided Person Search via Color Reasoning (119 citations)
- Attention Is Not Enough: Mitigating the Distribution Discrepancy in Asynchronous Multimodal Sequence Fusion (101 citations)
- Contrastive Multimodal Fusion With TupleInfoNCE (78 citations)
- Joint Representation Learning and Novel Category Discovery on Single- and Multi-Modal Data (64 citations)
- AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition (61 citations)
- MAAS: Multi-Modal Assignation for Active Speaker Detection (57 citations)
- Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining (57 citations)
- Multi-Modality Associative Bridging Through Memory: Speech Sound Recollected From Face Video (55 citations)
- How To Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild (54 citations)
and 10 more in the interactive map.
Visual question answering · 15 papers
- Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images (450 citations)
- TRAR: Routing the Attention Spans in Transformer for Visual Question Answering (133 citations)
- Unshuffling Data for Improved Generalization in Visual Question Answering (122 citations)
- Greedy Gradient Ensemble for Robust Visual Question Answering (102 citations)
- Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering (92 citations)
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models (66 citations)
- HAIR: Hierarchical Visual-Semantic Relational Reasoning for Video Question Answering (56 citations)
- Auto-Parsing Network for Image Captioning and Visual Question Answering (39 citations)
- Contrast and Classify: Training Robust VQA Models (34 citations)
- TransView: Inside, Outside, and Across the Cropping View Boundaries (27 citations)
- Weakly Supervised Relative Spatial Reasoning for Visual Question Answering (20 citations)
- Linguistically Routing Capsule Network for Out-of-Distribution Visual Question Answering (16 citations)
and 3 more in the interactive map.
Robot action and navigation · 6 papers
- Episodic Transformer for Vision-and-Language Navigation (160 citations)
- Airbert: In-Domain Pretraining for Vision-and-Language Navigation (158 citations)
- Waypoint Models for Instruction-Guided Navigation in Continuous Environments (109 citations)
- The Road To Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation (89 citations)
- Vision-Language Navigation With Random Environmental Mixup (89 citations)
- YouRefIt: Embodied Reference Understanding With Language and Gesture (41 citations)
Multimodal LLMs · 4 papers
- Panoptic Narrative Grounding (23 citations)
- Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision (16 citations)
- Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images (13 citations)
- When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes (13 citations)
