Research map / Vision-language and multimodal / Cross-modal learning
CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks
Wish Suharitdamrong, Tony Alex, Muhammad Awais, Sara Atito
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add inter-modal fusion pathway to LoRA (CoLA r=16 vs LoRA r=16) | dual-encoder PEFT · REC avg RefCOCO/+/g | accuracy avg (%): 82.3 -> 83.4 | Improved |
| add inter-modal pathway vs parameter-matched LoRA (r=54) | dual-encoder PEFT · REC avg | accuracy avg (%): 81.8 -> 83.4 | Improved |
| add inter-modal pathway to LoRA | dual-encoder PEFT · RES avg | mIoU avg: 72.2 -> 73.7 | Improved |
| raise LoRA rank from 16 to 54 to match parameters | LoRA · REC avg | accuracy avg (%): 82.3 -> 81.8 | Hurt |
| raise LoRA rank from 16 to 54 | LoRA · AVE | accuracy (%): 79.2 -> 79.2 | No clear effect |
| add CoLA over LoRA | dual-encoder PEFT · AVE | accuracy (%): 79.2 -> 80.7 | Improved |
| add CoLA over LoRA | dual-encoder PEFT · AVS | mIoU: 80.1 -> 80.9 | Improved |
| use non-shared low-rank matrices between intra and inter pathways instead of fully shared (ablation) | CoLA · RefCOCOg | REC avg: 81.2 -> 81.7 | Improved |
| use progressive cross-modal propagation instead of uniform or module-wise (ablation) | CoLA · RefCOCOg | avg REC+RES: 76.1 -> 76.5 | Improved |
| module-wise instead of uniform propagation (ablation) | CoLA · RefCOCOg | avg REC+RES: 76.1 -> 76.1 | No clear effect |
| use Kronecker factorization of Phi at symmetric rank (ablation) | CoLA · AVE | accuracy (%): 80.7 -> 79.9 | Hurt |
| add asymmetric ranks (intra 8, inter 32) with Kronecker Phi (ablation) | CoLA · AVE | accuracy (%): 80.7 -> 81.44 | Improved |
Closest papers by meaning
- A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter (AAAI 2025 · similarity 0.89)
- MokA: Multimodal Low-Rank Adaptation for MLLMs (NeurIPS 2025 · similarity 0.89)
- MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search (NeurIPS 2026 · similarity 0.88)
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.88)
- SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation (NeurIPS 2026 · similarity 0.88)
- Multimodal Representation Learning by Alternating Unimodal Adaptation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
