AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Contrastive Multimodal Fusion With TupleInfoNCE

Yunze Liu, Qingnan Fan, Shanghang Zhang, Hao Dong, Thomas Funkhouser, Li Yi

ICCV 2021 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 78 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add sample optimization of negative ratios (vs uniform alpha) (ablation)TupleInfoNCE · NYUv2 semantic segmentationmIoU: 1.7% mIoU drop without itImproved
add sample optimization of negative ratios (vs uniform alpha) (ablation)TupleInfoNCE · SUN RGB-D 3D detectionmAP@0.25: 0.5 mAP drop without itImproved
add sample optimization of negative ratios (vs uniform alpha) (ablation)TupleInfoNCE · CMU-MOSIAcc-7: 0.6 Acc-7 drop without itImproved
add sample optimization of negative ratios (vs uniform alpha) (ablation)TupleInfoNCE · CMU-MOSEIAcc-7: 0.4 Acc-7 drop without itImproved
optimize augmentation (positive samples) rather than hand-designed (ablation)TupleInfoNCE · NYUv2 semantic segmentationmIoU: 1.1 mIoU drop without itImproved
optimize augmentation (positive samples) rather than hand-designed (ablation)TupleInfoNCE · SUN RGB-D 3D detectionmAP@0.25: 0.6 mAP drop without itImproved
add tuple disturbing, sample optimization over plain InfoNCE on tuplesInfoNCE · NYUv2 semantic segmentationmIoU: 42.1 -> 48.1Improved
add tuple disturbing, sample optimization over plain InfoNCE on tuplesInfoNCE · SUN RGB-D 3D detectionmAP@0.25: 56.8 -> 58.0Improved
add depth and normal modalities to RGB (ablation)TupleInfoNCE · NYUv2 semantic segmentationmIoU: 40.8 -> 48.1Improved
add a fourth modality set (L, ab, depth, normal) (ablation)TupleInfoNCE · NYUv2 semantic segmentationmIoU: 48.1 -> 48.6Improved
add an uninformative timestamp modality (ablation)TupleInfoNCE · CMU-MOSEIAcc-7: 52.7 -> 52.6No clear effect
use InfoMin-searched rotation (70 degrees) instead of TupleInfoNCE-optimized (40 degrees) (ablation)augmentation search · NYUv2 semantic segmentationmIoU: 47.41 -> 46.91Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy