Research map / Vision-language and multimodal / Cross-modal learning
CoMIR: Contrastive Multimodal Image Representation for Registration
Nicolas Pielawski, Elisabeth Wetzer, Johan Öfverstedt, Jiahao Lu, Carolina Wählby, Joakim Lindblad, Natasa Sladoje
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| represent aligned BF and SHG images as CoMIRs, then register with SIFT | Biomedical BF–SHG registration · Biomedical test set, <1% error (<9px) | successful registrations: 75 [63; 86] vs 22 [14; 32] | Improved |
| use MSE rather than cosine similarity as the critic (ablation) | CoMIR registration with SIFT · Biomedical test set, <1% error (<9px) | successful registrations: 47 [36; 59] -> 75 [63; 86] | Improved |
| use MSE rather than cosine similarity as the critic (ablation) | CoMIR registration with α-AMD · Biomedical test set, <1% error (<9px) | successful registrations: 19 [12; 28] -> 49 [38; 61] | Improved |
| use cosine rather than MSE as the critic (ablation) | CoMIR representation learning · Zurich RGB–NIR | mean CoMIR correlation: 85.4% [83.2%; 88.1%] -> 91.5% [88.4%; 94.7%] | Improved |
| replace the symmetric MSE critic with a bilinear critic (ablation) | CoMIR representation learning · Zurich RGB–NIR | mean CoMIR correlation: 85.4% [83.2%; 88.1%] -> 60.9% [56.7%; 65.0%] | Hurt |
| use random rather than fixed-seed weight initialization (ablation) | CoMIR training on a fixed Zurich image · Zurich, fixed training image | mean pairwise CoMIR correlation: 93.0% [ 90.6%; 95.5%], N=100 -> -2.7% [-22.3%; -1.4%], N=40 | Hurt |
| enforce rotational equivariance using the C4-constrained contrastive loss (ablation) | CoMIR representations · Zurich RGB–NIR, rotations across [0, 360) | rotational equivariance | Improved |
| translate modalities with GAN-based image-to-image methods instead of learning CoMIRs | Biomedical BF–SHG registration with SIFT · Biomedical test set, <1% error (<9px) | successful registrations: 0 [0; 4] vs 75 [63; 86] | Hurt |
Closest papers by meaning
- Foundation-Guided Representation Alignment for Multimodal Medical Image Registration (ECCV 2026 · similarity 0.86)
- Multiple Modalities Image Matching With Large-Scale Pre-Training (TPAMI 2026 · similarity 0.86)
- Sparse-to-dense Multimodal Image Registration via Multi-Task Learning (ICML 2024 · similarity 0.86)
- Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration (CVPR 2024 · similarity 0.85)
- ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining (NeurIPS 2026 · similarity 0.84)
- CARL: A Framework for Equivariant Image Registration (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
