CVPR 2017 accepted papers on Vision-language and multimodal
53 of the 783 papers accepted at CVPR 2017 (6.8%) are on Vision-language and multimodal, compared with 5.1% at CVPR 2016. They fall into 4 topics.
See CVPR 2017 in the interactive map
Submitting to CVPR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Human and embodied modeling · 29 papers
- Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning (1,734 citations)
- Person Search With Natural Language Description (534 citations)
- Detecting Visual Relationships With Deep Relational Networks (507 citations)
- Semantic Compositional Networks for Visual Captioning (487 citations)
- Modeling Relationships in Referential Expressions With Compositional Modular Networks (410 citations)
- A Hierarchical Approach for Generating Descriptive Image Paragraphs (386 citations)
- MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network (316 citations)
- A Joint Speaker-Listener-Reinforcer Model for Referring Expressions (284 citations)
- Learning to Extract Semantic Structure From Documents Using Multimodal Fully Convolutional Neural Networks (255 citations)
- Instance-Aware Image and Sentence Matching With Selective Multimodal LSTM (254 citations)
- Emotion Recognition in Context (207 citations)
- Dense Captioning With Joint Inference and Visual Context (175 citations)
and 17 more in the interactive map.
Visual question answering · 13 papers
- Making the v in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering (2,316 citations)
- CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning (1,831 citations)
- Dual Attention Networks for Multimodal Reasoning and Matching (753 citations)
- Graph-Structured Representations for Visual Question Answering (494 citations)
- Visual Dialog (486 citations)
- TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering (480 citations)
- GuessWhat?! Visual Object Discovery Through Multi-Modal Dialogue (407 citations)
- Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension (258 citations)
- Multi-Level Attention Networks for Visual Question Answering (244 citations)
- The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions (97 citations)
- Knowledge Acquisition for Visual Question Answering via Iterative Querying (51 citations)
- An Empirical Evaluation of Visual Question Answering for Novel Objects (23 citations)
and 1 more in the interactive map.
Cross-modal learning · 9 papers
- Deep Cross-Modal Hashing (898 citations)
- Learning Cross-Modal Embeddings for Cooking Recipes and Food Images (421 citations)
- Cross-Modality Binary Code Learning via Fusion Similarity Hashing (260 citations)
- Learning Cross-Modal Deep Representations for Robust Pedestrian Detection (241 citations)
- Missing Modalities Imputation via Cascaded Residual Autoencoder (241 citations)
- Generalized Semantic Preserving Hashing for N-Label Cross-Modal Retrieval (132 citations)
- Online Asymmetric Similarity Learning for Cross-Modal Retrieval (44 citations)
- Hierarchical Multimodal Metric Learning for Multimodal Classification (39 citations)
- AMC: Attention guided Multi-modal Correlation Learning for Image Search (37 citations)
CLIP and zero-shot · 2 papers
- Learning a Deep Embedding Model for Zero-Shot Learning (792 citations)
- Link the Head to the "Beak": Zero Shot Learning From Noisy Text Description at Part Precision (129 citations)
