Research map / Vision-language and multimodal / Cross-modal learning
iQuery: Instruments As Queries for Audio-Visual Sound Separation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use iQuery instead of CCoL | Audio-visual separation · MUSIC test | SDR: 7.74 -> 11.17 | Helped |
| Use iQuery instead of AMnet | Audio-visual separation · MUSIC-21 | SDR: 11.08 -> 11.12 | Helped |
| Use iQuery instead of AMnet | Audio-visual separation · AVE | SDR: 3.71 -> 5.02 | Helped |
| Fine-tune a new audio prompt while freezing the transformer backbone rather than fine-tuning the whole network | MUSIC held-out instrument adaptation · MUSIC held-out instrument fine-tuning | SDR: 6.74 -> 8.04 | Helped |
| Add the learnable linear layer to decoder queries (ablation) | iQuery with auxiliary contrastive-loss setup · MUSIC | SDR: 10.05 -> 11.17 | Helped |
| Use hierarchical task learning rather than a fixed contrastive-loss weight (ablation) | iQuery with auxiliary contrastive loss · MUSIC | SDR: 10.89 -> 11.02 | Helped |
| Add auxiliary contrastive loss (ablation) | iQuery · MUSIC | SDR: 11.17 -> 11.02 | Hurt |
| Use visually named query assignments rather than random object-feature assignments (ablation) | iQuery decoder · MUSIC | SDR: 6.58 -> 11.17 | Helped |
| Add motion cross-attention (ablation) | Self-audio iQuery decoder · MUSIC | SDR: 10.54 -> 11.17 | Helped |
| Combine self-, motion-cross-, and audio-cross-attention in one decoder layer instead of the selected staged decoder (ablation) | iQuery decoder · MUSIC | SDR: 11.17 -> 10.65 | Hurt |
| Run motion and audio cross-attention in parallel and fuse them instead of the selected staged decoder (ablation) | iQuery decoder · MUSIC | SDR: 11.17 -> 10.46 | Hurt |
Closest papers by meaning
- OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup (ICLR 2025 · similarity 0.88)
- Continual Audio-Visual Sound Separation (NeurIPS 2024 · similarity 0.88)
- Independency Adversarial Learning for Cross-Modal Sound Separation (AAAI 2024 · similarity 0.86)
- X-AVDD: Cross-Attentive Audio-Visual Dataset Distillation (NeurIPS 2026 · similarity 0.86)
- Weakly-supervised Audio Separation via Bi-modal Semantic Similarity (ICLR 2024 · similarity 0.86)
- T-VSL: Text-Guided Visual Sound Source Localization in Mixtures (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.