Research map / Vision-language and multimodal / Cross-modal learning
PROMISE: Prompt-Attentive Hierarchical Contrastive Learning for Robust Cross-Modal Representation with Missing Modalities
Jiajun Chen, Sai Cheng, Yuan Yutao, YiRui Zhang, Haitao Yuan, Peng Peng, Yi Zhong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add both FNCL and CCCL contrastive losses on top of Prompt Attention (ablation) | PROMISE · N24News | ACC (%): 66.31 -> 68.89 | Improved |
| add both FNCL and CCCL contrastive losses on top of Prompt Attention (ablation) | PROMISE · Hateful Memes | ACC (%): 57.11 -> 65.47 | Improved |
| add Prompt Attention without contrastive losses (ablation) | baseline without prompt or contrastive · Hateful Memes | ACC (%): 56.89 -> 57.11 | Improved |
| add Prompt Attention without contrastive losses (ablation) | baseline without prompt or contrastive · N24News | ACC (%): 64.01 -> 66.31 | Improved |
| add only one of FNCL or CCCL (ablation) | PROMISE · Hateful Memes / N24News | ACC (%) | Mixed |
| use CCCL only (intra-modal) instead of MPVR baseline (ablation) | PROMISE-CCCL · N24News missing-image/text | accuracy | Improved |
| add FNCL when an entire modality is absent (ablation) | PROMISE · N24News missing-both | accuracy | Improved |
| increase prompt-attention layers beyond 6 (7-8 layers) (ablation) | PROMISE · Hateful Memes | AUROC: 67.17% | No clear effect |
| increase prompt dimension beyond 48 (ablation) | PROMISE · Hateful Memes | AUROC | Hurt |
| use 1 or 2 prompt-attention layers (ablation) | PROMISE · Hateful Memes | AUROC | Hurt |
Closest papers by meaning
- Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning (AAAI 2025 · similarity 0.91)
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.88)
- Do More Modalities Always Help? A Geometric Perspective on Missing-Modality Robustness (NeurIPS 2026 · similarity 0.88)
- RedCore: Relative Advantage Aware Cross-Modal Representation Learning for Missing Modalities with Imbalanced Missing Rates (AAAI 2024 · similarity 0.87)
- Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning (NeurIPS 2024 · similarity 0.87)
- Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
