AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

CoMIR: Contrastive Multimodal Image Representation for Registration

Nicolas Pielawski, Elisabeth Wetzer, Johan Öfverstedt, Jiahao Lu, Carolina Wählby, Joakim Lindblad, Natasa Sladoje

NeurIPS 2020 · Vision-language and multimodal › Cross-modal learning · approach: Multi-modality image fusion · 59 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
represent aligned BF and SHG images as CoMIRs, then register with SIFTBiomedical BF–SHG registration · Biomedical test set, <1% error (<9px)successful registrations: 75 [63; 86] vs 22 [14; 32]Improved
use MSE rather than cosine similarity as the critic (ablation)CoMIR registration with SIFT · Biomedical test set, <1% error (<9px)successful registrations: 47 [36; 59] -> 75 [63; 86]Improved
use MSE rather than cosine similarity as the critic (ablation)CoMIR registration with α-AMD · Biomedical test set, <1% error (<9px)successful registrations: 19 [12; 28] -> 49 [38; 61]Improved
use cosine rather than MSE as the critic (ablation)CoMIR representation learning · Zurich RGB–NIRmean CoMIR correlation: 85.4% [83.2%; 88.1%] -> 91.5% [88.4%; 94.7%]Improved
replace the symmetric MSE critic with a bilinear critic (ablation)CoMIR representation learning · Zurich RGB–NIRmean CoMIR correlation: 85.4% [83.2%; 88.1%] -> 60.9% [56.7%; 65.0%]Hurt
use random rather than fixed-seed weight initialization (ablation)CoMIR training on a fixed Zurich image · Zurich, fixed training imagemean pairwise CoMIR correlation: 93.0% [ 90.6%; 95.5%], N=100 -> -2.7% [-22.3%; -1.4%], N=40Hurt
enforce rotational equivariance using the C4-constrained contrastive loss (ablation)CoMIR representations · Zurich RGB–NIR, rotations across [0, 360)rotational equivarianceImproved
translate modalities with GAN-based image-to-image methods instead of learning CoMIRsBiomedical BF–SHG registration with SIFT · Biomedical test set, <1% error (<9px)successful registrations: 0 [0; 4] vs 75 [63; 86]Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy