Research map / Vision-language and multimodal / Cross-modal learning
Decoupled and Reusable Adaptation for Efficient Cross-Modal Transfer
Yajing Liu, Yumeng Zhang, Yue Si, Baojie Fan, Jiandong Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Stage 1 Progressive Self-Supervised Tuning (modality LoRA) (ablation) | TP-MoME with SAM 2 Hiera-B+ · MCubeS and DELIVER (average) | Avg mIoU (%): 54.9 -> 59.3 | Improved |
| add MAE training in Stage 1 (ablation) | TP-MoME · MCubeS and DELIVER (average) | Avg mIoU (%): 54.9 -> 56.2 | Improved |
| add DINOv2-style training in Stage 1 (ablation) | TP-MoME · MCubeS and DELIVER (average) | Avg mIoU (%): 54.9 -> 57.1 | Improved |
| combine MAE and DINOv2 trained independently (ablation) | TP-MoME · MCubeS and DELIVER (average) | Avg mIoU (%): 54.9 -> 58.2 | Improved |
| use MAE warm-up then MAE-guided consistency learning instead of independent training (ablation) | PSST · MCubeS and DELIVER (average) | Avg mIoU (%): 58.2 -> 59.3 | Improved |
| use MoME instead of simple gated summation for fusion (ablation) | TP-MoME · MCubeS and DELIVER (average) | Avg mIoU (%): 58.1 -> 59.3 | Improved |
| use MoME instead of gated summation with more modalities (RGB-A-D) (ablation) | TP-MoME · MCubeS (RGB-A-D) | mIoU (%): 52.7 -> 54.4 | Improved |
| use MoME instead of gated summation with more modalities (RGB-D-E) (ablation) | TP-MoME · DELIVER (RGB-Depth-Event) | mIoU (%): 63.6 -> 65.1 | Improved |
| decoupled modality LoRA plus task adaptation versus MemorySAM on trainable parameters | Ours · Five downstream tasks (storage) | Accumulated parameters (M): 153.7 -> 121.4 | Improved |
| add Stage 1 modality knowledge, label efficiency | Ours · MCubeS | mIoU | Improved |
| add Stage 1 modality knowledge, training speed (ablation) | Ours · MCubeS | training epochs: epoch 5 vs MemorySAM epoch 12 | Improved |
| Ours (Hiera-B+) versus full fine-tuned CMNeXt on RGB-T MFNet | Ours · MFNet (RGB-T) | mIoU (%): 59.9 -> 60.5 | Improved |
Closest papers by meaning
- Learning Modality Knowledge Alignment for Cross-Modality Transfer (ICML 2024 · similarity 0.90)
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.89)
- ReconBoost: Boosting Can Achieve Modality Reconcilement (ICML 2024 · similarity 0.89)
- Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs (NeurIPS 2026 · similarity 0.88)
- Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge (NeurIPS 2025 · similarity 0.87)
- Keep the Balance: A Parameter-Efficient Symmetrical Framework for RGB+X Semantic Segmentation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
