Research map / Vision-language and multimodal / Cross-modal learning
Aligning the True Semantics: Constrained Decoupling and Distribution Sampling for Cross-Modal Alignment
Xiang Ma, Lexin Fang, Litian Xu, Caiming Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add decoupling architecture (dual-path UNet) (ablation) | CDDS · Flickr30K I-to-T R@1 (ViT-224) | R@1: 71.4 -> 74.8 | Improved |
| add information integrity (reconstruction) constraint (ablation) | CDDS · Flickr30K I-to-T R@1 (ViT-224) | R@1: 69.8 -> 74.8 | Improved |
| add modal component constraint (ablation) | CDDS · Flickr30K I-to-T R@1 (ViT-224) | R@1: 73.8 -> 74.8 | Improved |
| add modal component constraint (ablation) | CDDS · Flickr30K I-to-T R@1 (Swin-224) | R@1: 80.4 -> 83.0 | Improved |
| add decoupling architecture (ablation) | CDDS · Flickr30K I-to-T R@1 (Swin-224) | R@1: 82.2 -> 83.0 | Improved |
| add Gaussian noise to high-dimensional representations (ablation) | CDDS · Flickr30K I-to-T R@1 (ViT-224) | R@1: 74.2 -> 74.8 | Improved |
| use distribution sampling instead of contrastive learning for semantic alignment (ablation) | CDDS · Flickr30K I-to-T R@1 (ViT-224) | R@1: 72.1 -> 74.8 | Improved |
| add distribution sampling to existing models (VSE++, SCAN, SGR, CHAN, LAPS) (ablation) | existing retrieval models · Flickr30K / MS-COCO 5K (ViT-224) | R@K: +0.4% to +1.1% | Improved |
| apply CDDS to CLIP backbone versus zero-shot CLIP | CLIP ViT-224 + BERT · Flickr30K image-to-text R@1 | R@1: 81.4 -> 93.5 | Improved |
| approximate Eq.5 by random feature selection instead of per-batch computation (ablation) | CDDS · Flickr30K rSum | rSum: 510.6 -> 502.3 | Hurt |
| compute Eq.5 once on the total dataset before training (ablation) | CDDS · MS-COCO 5K rSum | rSum: 437.8 -> 411.6 | Hurt |
| CDDS still below large VLP models | CDDS with CLIP · Flickr30K image-to-text R@1 | R@1: 93.5 vs BLIP 96.6 | No clear effect |
Closest papers by meaning
- Linguistic-Aware Patch Slimming Framework for Fine-grained Cross-Modal Alignment (CVPR 2024 · similarity 0.90)
- Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs (NeurIPS 2026 · similarity 0.90)
- On the Value of Cross-Modal Misalignment in Multimodal Representation Learning (NeurIPS 2025 · similarity 0.89)
- Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion (ICLR 2025 · similarity 0.88)
- Guiding Cross-Modal Representations with MLLM Priors via Preference Alignment (NeurIPS 2025 · similarity 0.88)
- Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
