AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

MultiMem: Measuring and Mitigating Memorization in Multi-Modal Contrastive Learning

Wenhao Wang, Franziska Boenisch, Michael Backes, Adam Dziedzic

ECCV 2026 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add noise augmentation to all three modalities instead of one (ablation)AudioCLIP · AudioCLIP UrbanSound8KAIT MultiMem / AI-T T@5: 0.332 -> 0.264 MultiMem; 36.9 -> 43.1 T@5Improved
apply noise to a single modality only (ablation)AudioCLIP · AudioCLIP retrievalT@5 on unaugmented pairs: I-T 33.1 -> 32.2 when audio augmentedMixed
in-training noise on top 5% MultiMem-memorized samplesAudioCLIP · AudioCLIP UrbanSound8K retrievalretrieval T@5: 36.9% -> 43.3%Improved
post-training remove most memorized samples and fine-tuneAudioCLIP · AudioCLIP UrbanSound8K retrievalretrieval T@5: 36.9% -> 39.3%Improved
select samples by MultiMem rather than loss or gradient (ablation)in-training noise mitigation · AudioCLIP retrievalretrieval T@5: 39.9% (loss) -> 43.3%Improved
add noise to random samples (ablation)in-training mitigation · AudioCLIP retrievalretrieval T@5: 36.9% -> 37.2%No clear effect
add noise to least memorized samples (ablation)in-training mitigation · AudioCLIP retrievalretrieval T@5: 36.9% -> 36.1%Hurt
modality-specific augmentations instead of Gaussian noise (ablation)in-training mitigation · AudioCLIP retrievalretrieval T@5: 43.3% -> 42.5%No clear effect
ImageBind loss instead of pairwise alignment of all modalities (ablation)AVIT-CLIP · MSR-VTT AVITMultiMem / retrieval: 0.488 -> 0.571 MultiMem; 48.2% -> 36.5% retrievalHurt
balance per-pair loss weights negatively correlated to memorization (ablation)AudioCLIP · AudioCLIPMultiMem: 0.332 -> 0.290Improved
vary candidate set size S_C from 500 to 2000 (ablation)MultiMem metric · AVT-CLIP MSR-VTTaverage MultiMem: 0.405 -> 0.408No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy