Research map / Vision-language and multimodal / Cross-modal learning
Learning Aligned Cross-Modal Representations From Weakly Aligned Data
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply modality tuning rather than ordinary finetuning | Places-CNN cross-modal representation · CMPlaces cross-modal retrieval | mAP, fc7: 6.4 -> 10.5 | Helped |
| Combine modality tuning with GMM activation regularization | Places-CNN cross-modal representation · CMPlaces cross-modal retrieval | mAP, fc7: 10.5 -> 14.2 | Helped |
| Use a GMM rather than a single Gaussian for activation regularization (ablation) | StatReg · CMPlaces cross-modal retrieval | mAP, fc7: 7.2 -> 9.5 | Helped |
| Use a GMM rather than a single Gaussian for activation regularization (ablation) | StatReg · CMPlaces within-modality retrieval | mean mAP: 32.6 -> 32.6 | No clear effect |
| Unfreeze shared layers during modality tuning (ablation) | A: Tune · CMPlaces within-modality retrieval | mean mAP: 33.4 -> 35.0 | Helped |
| Unfreeze shared layers during modality tuning (ablation) | A: Tune · CMPlaces cross-modal retrieval | mAP, fc7: 10.5 -> 11.2 | Helped |
| Add modality tuning and GMM statistical regularization to the representation | Places-CNN cross-modal representation · CMPlaces within-modality retrieval | mean mAP: 32.6 -> 35.0 | Helped |
Closest papers by meaning
- Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (NeurIPS 2026 · similarity 0.89)
- CrossOver: 3D Scene Cross-Modal Alignment (CVPR 2025 · similarity 0.89)
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.88)
- Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion (ICLR 2025 · similarity 0.88)
- CrossMAE: Cross-Modality Masked Autoencoders for Region-Aware Audio-Visual Pre-Training (CVPR 2024 · similarity 0.87)
- Learning Modality Knowledge Alignment for Cross-Modality Transfer (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.