AAAI 2022 accepted papers on Vision-language and multimodal
60 of the 1,319 papers accepted at AAAI 2022 (4.5%) are on Vision-language and multimodal, compared with 4.1% at AAAI 2021. They fall into 5 topics.
See AAAI 2022 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Human and embodied modeling · 25 papers
- End-to-End Transformer Based Model for Image Captioning (154 citations)
- BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents (129 citations)
- Clinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation (88 citations)
- Context-Based Contrastive Learning for Scene Text Recognition (59 citations)
- Image Difference Captioning with Pre-training and Contrastive Learning (53 citations)
- Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition (53 citations)
- UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation (43 citations)
- NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-Task Financial Forecasting (41 citations)
- Action-Aware Embedding Enhancement for Image-Text Retrieval (37 citations)
- SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning (37 citations)
- DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents (36 citations)
- Attention-Aligned Transformer for Image Captioning (35 citations)
and 13 more in the interactive map.
Cross-modal learning · 18 papers
- AXM-Net: Implicit Cross-Modal Feature Alignment for Person Re-identification (140 citations)
- Show Your Faith: Cross-Modal Confidence-Aware Network for Image-Text Matching (80 citations)
- Tailor Versatile Multi-Modal Learning for Multi-Label Emotion Recognition (75 citations)
- Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading (68 citations)
- Interact, Embed, and EnlargE: Boosting Modality-Specific Representations for Multi-Modal Person Re-identification (54 citations)
- Hierarchical Cross-Modality Semantic Correlation Learning Model for Multimodal Summarization (47 citations)
- MuMu: Cooperative Multitask Learning-Based Guided Multimodal Fusion (41 citations)
- Towards Versatile Pedestrian Detector with Multisensory-Matching and Multispectral Recalling Memory (37 citations)
- Promoting Single-Modal Optical Flow Network for Diverse Cross-Modal Flow Estimation (31 citations)
- Multi-Modal Perception Attention Network with Self-Supervised Learning for Audio-Visual Speaker Tracking (30 citations)
- Visual Sound Localization in the Wild by Cross-Modal Interference Erasing (25 citations)
- Cross-Modal Mutual Learning for Audio-Visual Speech Recognition and Manipulation (15 citations)
and 6 more in the interactive map.
Visual question answering · 9 papers
- An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA (284 citations)
- Multi-Modal Answer Validation for Knowledge-Based VQA (115 citations)
- Video as Conditional Graph Hierarchy for Multi-Granular Question Answering (107 citations)
- Playing Lottery Tickets with Vision and Language (32 citations)
- Dynamic Key-Value Memory Enhanced Multi-Step Graph Reasoning for Knowledge-Based Visual Question Answering (22 citations)
- Zero-Shot Commonsense Question Answering with Cloze Translation and Consistency Optimization (21 citations)
- MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding (20 citations)
- Generation-Focused Table-Based Intermediate Pre-training for Free-Form Question Answering (9 citations)
- Learning the Dynamics of Visual Relational Reasoning via Reinforced Path Routing (5 citations)
CLIP and zero-shot · 7 papers
- TransZero: Attribute-Guided Transformer for Zero-Shot Learning (161 citations)
- Open Vocabulary Electroencephalography-to-Text Decoding and Zero-Shot Sentiment Classification (66 citations)
- TVT: Three-Way Vision Transformer through Multi-Modal Hypersphere Learning for Zero-Shot Sketch-Based Image Retrieval (56 citations)
- DKPLM: Decomposable Knowledge-Enhanced Pre-trained Language Model for Natural Language Understanding (36 citations)
- Learning Aligned Cross-Modal Representation for Generalized Zero-Shot Classification (31 citations)
- Language Model Priming for Cross-Lingual Event Extraction (28 citations)
- Improving Zero-Shot Phrase Grounding via Reasoning on External Knowledge and Spatial Relations (9 citations)
