ECCV 2022 accepted papers on Vision-language and multimodal
80 of the 1,645 papers accepted at ECCV 2022 (4.9%) are on Vision-language and multimodal, compared with 5.1% at ECCV 2020. They fall into 6 topics.
See ECCV 2022 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Human and embodied modeling · 29 papers
- OCR-Free Document Understanding Transformer (734 citations)
- Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing (396 citations)
- SeqTR: A Simple Yet Universal Network for Visual Grounding (178 citations)
- UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling (110 citations)
- Multi-Granularity Prediction for Scene Text Recognition (94 citations)
- Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognition (92 citations)
- A Simple and Robust Correlation Filtering Method for Text-Based Person Search (85 citations)
- Emotion Recognition for Multiple Context Awareness (72 citations)
- "“This Is My Unicorn, Fluffy”: Personalizing Frozen Vision-Language Representations" (70 citations)
- D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding (62 citations)
- FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context (53 citations)
- Levenshtein OCR (45 citations)
and 17 more in the interactive map.
Cross-modal learning · 19 papers
- MultiMAE: Multi-modal Multi-task Masked Autoencoders (334 citations)
- Audio—Visual Segmentation (168 citations)
- Cross-Modal Prototype Driven Network for Radiology Report Generation (119 citations)
- Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features (98 citations)
- Localizing Visual Sounds the Easy Way (74 citations)
- CMD: Self-Supervised 3D Action Representation Learning with Cross-Modal Mutual Distillation (70 citations)
- CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval (39 citations)
- Contrastive Vision-Language Pre-training with Limited Resources (30 citations)
- Speaker-Adaptive Lip Reading with User-Dependent Padding (28 citations)
- Uncertainty-Aware Multi-modal Learning via Cross-Modal Random Network Prediction (28 citations)
- AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation (25 citations)
- Active Audio-Visual Separation of Dynamic Sound Sources (22 citations)
and 7 more in the interactive map.
CLIP and zero-shot · 12 papers
- Tip-Adapter: Training-Free Adaption of CLIP for Few-Shot Classification (522 citations)
- Joint Learning of Localized Representations from Medical Images and Reports (95 citations)
- MVP: Multimodality-Guided Visual Pre-training (90 citations)
- Zero-Shot Temporal Action Detection via Vision-Language Prompting (72 citations)
- Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training (53 citations)
- FashionViL: Fashion-Focused Vision-and-Language Representation Learning (51 citations)
- Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting (43 citations)
- PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks (28 citations)
- No Token Left Behind: Explainability-Aided Image Classification and Generation (27 citations)
- Generative Negative Text Replay for Continual Vision-Language Pretraining (22 citations)
- Single-Stream Multi-level Alignment for Vision-Language Pretraining (16 citations)
- GRIT-VLP: Grouped Mini-Batch Sampling for Efficient Vision and Language Pre-training
Visual question answering · 7 papers
- A-OKVQA: A Benchmark for Visual Question Answering Using World Knowledge (414 citations)
- Rethinking Data Augmentation for Robust Visual Question Answering (66 citations)
- Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly (50 citations)
- AssistQ: Affordance-Centric Question-Driven Task Completion for Egocentric Assistant (23 citations)
- Weakly Supervised Grounding for VQA in Vision-Language Transformers (19 citations)
- Classification-Regression for Chart Comprehension (15 citations)
- New Datasets and Models for Contextual Reasoning in Visual Dialog (1 citation)
Robot action and navigation · 7 papers
- Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environments (71 citations)
- A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility (49 citations)
- Learning from Unlabeled 3D Environments for Vision-and-Language Navigation (46 citations)
- Multimodal Transformer with Variable-Length Memory for Vision-and-Language Navigation (37 citations)
- ASSISTER: Assistive Navigation via Conditional Instruction Generation (15 citations)
- Learning Disentanglement with Decoupled Labels for Vision-Language Navigation (11 citations)
- Bridging the Visual Semantic Gap in VLN via Semantically Richer Instructions (2 citations)
Multimodal LLMs · 6 papers
- The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning (37 citations)
- SiRi: A Simple Selective Retraining Mechanism for Transformer-Based Visual Grounding (24 citations)
- Benchmarking Omni-Vision Representation through the Lens of Visual Realms (21 citations)
- PACS: A Dataset for Physical Audiovisual Commonsense Reasoning (18 citations)
- Spatial and Visual Perspective-Taking via View Rotation and Relation Reasoning for Embodied Reference Understanding (16 citations)
- Fine-Grained Visual Entailment (7 citations)
