Research map / Vision-language and multimodal / Cross-modal learning
Beyond Instance-Level Alignment: Dual-Level Optimal Transport for Audio-Text Retrieval
Wenqi Guo, Shikui Tu, Lei Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reliability-aware marginals (RAM) (ablation) | DART (ResNet38+BERT) · AudioCaps | audio-to-text R@1: 54.44 -> 55.27 | Improved |
| add reliability-aware marginals (RAM) (ablation) | DART (BEATs+BERT) · AudioCaps | audio-to-text R@1: 71.1 -> 72.1 | Improved |
| add reliability-aware marginals (RAM) (ablation) | DART (ResNet38+BERT) · Clotho | audio-to-text R@1: 22.48 -> 23.54 | Improved |
| add reliability-aware marginals (RAM) (ablation) | DART (ResNet38+BERT) · Clotho | text-to-audio R@1: 17.30 -> 17.18 | No clear effect |
| use correlation alone as marginal instead of full RAM (ablation) | DART · AudioCaps (batch 64) | mean R@1: 44.92 -> 44.35 | Hurt |
| use full RAM (corr, variance, kurtosis) instead of uniform marginals (ablation) | DART · AudioCaps (batch 64) | mean R@1: 44.92 -> 45.55 | Improved |
| use kurtosis-only marginals instead of uniform (ablation) | DART · AudioCaps (batch 64) | mean R@1: 44.92 -> 45.29 | Improved |
| add feature-level loss L_UWD to L_IOT (ablation) | DART · AudioCaps (batch 128) | audio-to-text R@1: 52.45 -> 53.08 | Improved |
| use L_UWD alone without instance-level loss (ablation) | DART · AudioCaps (batch 128) | audio-to-text R@1: 53.08 -> 0.10 | Hurt |
| replace matching/IOT loss with DART feature-level extension | IOT loss · ESC-50 zero-shot | R@1: 79.25 -> 80.75 | Improved |
| replace Luong et al. with DART under 40% unlabeled data | Luong et al. 2024 · AudioCaps semi-supervised 40% unlabeled | text-to-audio R@1: 28.58 -> 33.24 | Improved |
| replace Luong et al. with DART under 40% noisy captions | Luong et al. 2024 · AudioCaps noisy labels 40% | audio-to-text R@1: 34.37 -> 37.09 | Improved |
Closest papers by meaning
- Revisiting Deep Audio-Text Retrieval Through the Lens of Transportation (ICLR 2024 · similarity 0.89)
- Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval (CVPR 2024 · similarity 0.84)
- Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gap (NeurIPS 2026 · similarity 0.84)
- FLAM: Frame-Wise Language-Audio Modeling (ICML 2025 · similarity 0.83)
- With Limited Data for Multimodal Alignment, Let the STRUCTURE Guide You (NeurIPS 2025 · similarity 0.83)
- Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
