TPAMI 2024 papers on Vision-language and multimodal
49 of the 666 papers published in TPAMI 2024 (7.4%) are on Vision-language and multimodal, compared with 5.0% at TPAMI 2023. They fall into 6 topics.
See TPAMI 2024 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Cross-modal learning · 21 papers
- Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption (260 citations)
- VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset (185 citations)
- Deep Learning for Visual Speech Analysis: A Survey (88 citations)
- Robust Multimodal Learning With Missing Modalities via Parameter-Efficient Adaptation (82 citations)
- Detecting and Grounding Multi-Modal Media Manipulation and Beyond (69 citations)
- Cross-Modal Hashing Method With Properties of Hamming Space: A New Perspective (58 citations)
- On-the-Fly Modulation for Balanced Multimodal Learning (46 citations)
- Probing Synergistic High-Order Interaction for Multi-Modal Image Fusion (46 citations)
- Causality-Invariant Interactive Mining for Cross-Modal Similarity Learning (39 citations)
- An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits (32 citations)
- Unsupervised Dual Deep Hashing With Semantic-Index and Content-Code for Cross-Modal Retrieval (30 citations)
- Language-Aware Vision Transformer for Referring Segmentation (29 citations)
and 9 more in the interactive map.
CLIP and zero-shot · 7 papers
- Vision-Language Models for Vision Tasks: A Survey (1,301 citations)
- Prompt Tuning of Deep Neural Networks for Speaker-Adaptive Visual Speech Recognition (34 citations)
- ZeroNLG: Aligning and Autoencoding Domains for Zero-Shot Multimodal and Multilingual Natural Language Generation (22 citations)
- Fine-Grained Visual Text Prompting (17 citations)
- Tuning Vision-Language Models With Multiple Prototypes Clustering (17 citations)
- Noise-Robust Vision-Language Pre-Training With Positive-Negative Learning (16 citations)
- Multi-Modality Multi-Attribute Contrastive Pre-Training for Image Aesthetics Computing (14 citations)
Human and embodied modeling · 7 papers
- SpectralGPT: Spectral Remote Sensing Foundation Model (1,117 citations)
- SMART: Syntax-Calibrated Multi-Aspect Relation Transformer for Change Captioning (67 citations)
- Unsupervised Illumination Adaptation for Low-Light Vision (32 citations)
- Efficient Visual Computing With Camera RAW Snapshots (15 citations)
- Unpaired Image-Text Matching via Multimodal Aligned Conceptual Knowledge (12 citations)
- Partial Scene Text Retrieval
- Rebuttal to “Comments on ‘Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features’ ”
Multimodal LLMs · 6 papers
- LVLM-EHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models (282 citations)
- Vision-Centric BEV Perception: A Survey (228 citations)
- Q-Bench+: A Benchmark for Multi-Modal Foundation Models on Low-Level Vision From Single Images to Pairs (60 citations)
- Vision + X: A Survey on Multimodal Learning in the Light of Data (56 citations)
- Divert More Attention to Vision-Language Object Tracking (19 citations)
- Every Problem, Every Step, All in Focus: Learning to Solve Vision-Language Problems With Integrated Attention (4 citations)
Visual question answering · 6 papers
- Robust Visual Question Answering: Datasets, Methods, and Future Challenges (64 citations)
- Towards Visual-Prompt Temporal Answer Grounding in Instructional Video (55 citations)
- Integrating Neural-Symbolic Reasoning With Variational Causal Inference Network for Explanatory Visual Question Answering (33 citations)
- UNK-VQA: A Dataset and a Probe Into the Abstention Ability of Multi-Modal Large Models (19 citations)
- Transformer Module Networks for Systematic Generalization in Visual Question Answering (12 citations)
- Ask Questions With Double Hints: Visual Question Generation With Answer-Awareness and Region-Reference (5 citations)
