Research map / Vision-language and multimodal / Cross-modal learning
Cross-Modal Redundancy and the Geometry of Vision–Language Embeddings
Grégoire DHIMOÏLA, Thomas Fel, Victor Boutin, Agustin Martin Picard
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE, CLIP · LAION | functional alignment rho: 0.327 -> 4.232 | Improved |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE, OpenCLIP · LAION | functional alignment rho: 4.072 -> 16.02 | Improved |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE, SigLIP · LAION | FDA: 8.831 -> 34.95 | Improved |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE, CLIP · LAION | probing accuracy: 0.847 -> 0.915 | Improved |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE, CLIP · LAION | interventional robustness delta_r: 0.224 -> 0.125 | Improved |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE, CLIP · LAION | reconstruction R2: 0.859 -> 0.837 | No clear effect |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE · synthetic, Iso-Energy holds | Wasserstein distance: 0.396 -> 0.184 | Improved |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE · synthetic, Iso-Energy holds | mean matching accuracy: 0.29 -> 0.52 | Improved |
| add Iso-Energy alignment loss (ablation) | Matching Pursuit SAE · synthetic, Iso-Energy violated | Wasserstein distance: 0.19 | No clear effect |
| restrict vector arithmetic to bimodal atoms | CLIP query · FashionIQ | OOD score: 0.97 -> 0.77 | Improved |
| restrict vector arithmetic to bimodal atoms | SigLIP2 query · FashionIQ | OOD score: 0.99 -> 0.61 | Improved |
| filter unimodal atoms with binary mask | CLIP embeddings · modality gap | modality gap | Improved |
Closest papers by meaning
- Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (NeurIPS 2026 · similarity 0.92)
- Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs (NeurIPS 2026 · similarity 0.90)
- Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment (NeurIPS 2024 · similarity 0.90)
- AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding (NeurIPS 2025 · similarity 0.90)
- The Alignment Illusion in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.90)
- Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
