ICML 2023 accepted papers on Vision-language and multimodal
41 of the 1,865 papers accepted at ICML 2023 (2.2%) are on Vision-language and multimodal, compared with 1.4% at ICML 2022. They fall into 5 topics.
See ICML 2023 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Human and embodied modeling · 13 papers
- Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding (47 citations)
- Retrieval-Augmented Multimodal Language Modeling (28 citations)
- Grounding Language Models to Images for Multimodal Inputs and Outputs (25 citations)
- TRAK: Attributing Model Behavior at Scale (9 citations)
- MEWL: Few-shot multimodal word learning with referential uncertainty (5 citations)
- Learning Dense Correspondences between Photos and Sketches (3 citations)
- On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline (2 citations)
- On the Complexity of Bayesian Generalization (1 citation)
- Towards credible visual model interpretation with path attribution (1 citation)
- Deep Temporal Sets with Evidential Reinforced Attentions for Unique Behavioral Pattern Discovery
- Evidential Interactive Learning for Medical Image Captioning
- Neural Prediction Errors enable Analogical Visual Reasoning in Human Standard Intelligence Tests
and 1 more in the interactive map.
CLIP and zero-shot · 11 papers
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models (917 citations)
- Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language (30 citations)
- ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts (30 citations)
- Improving Visual Prompt Tuning for Self-supervised Vision Transformers (7 citations)
- POUF: Prompt-Oriented Unsupervised Fine-tuning for Large Pre-trained Models (7 citations)
- Text-To-Concept (and Back) via Cross-Model Alignment (3 citations)
- Hyperbolic Image-text Representations (2 citations)
- Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning (2 citations)
- One-shot Imitation in a Non-Stationary Environment via Multi-Modal Skill (1 citation)
- $\pi$-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation
- On the Generalization of Multi-modal Contrastive Learning
Cross-modal learning · 11 papers
- mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video (49 citations)
- Provable Dynamic Fusion for Low-Quality Multimodal Data (20 citations)
- Calibrating Multimodal Learning (10 citations)
- On Uni-Modal Feature Learning in Supervised Multi-Modal Learning (9 citations)
- Scaling Laws for Generative Mixed-Modal Language Models (7 citations)
- A Unified Audio-Visual Learning Framework for Localization, Separation, and Recognition (6 citations)
- UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers (6 citations)
- Continual Vision-Language Representation Learning with Off-Diagonal Information (2 citations)
- Cross-Modal Fine-Tuning: Align then Refine (1 citation)
- Auxiliary Modality Learning with Generalized Curriculum Distillation
- Robustness in Multimodal Learning under Train-Test Modality Mismatch