Research map / Vision-language and multimodal / Cross-modal learning
Can Textual Semantics Mitigate Sounding Object Segmentation Preference?
Yaoting Wang, Peiwen Sun, Yuanchao Li, Honggang Zhang, Di Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SeDAM module for text-audio fusion (ablation) | TeSO · AVS V1M | mIoU (%): 62.22 -> 66.02 | Improved |
| add PMQS module for prompting mask queries (ablation) | TeSO · AVS V1M | mIoU (%): 62.90 -> 66.02 | Improved |
| add dynamic mask based on attention score (ablation) | TeSO · AVS V1M | mIoU (%): 64.78 -> 66.02 | Improved |
| add InfoNCE latent component loss (ablation) | TeSO · AVS V1M | mIoU (%): 65.34 -> 66.02 | Improved |
| add text guidance in addition to audio (ablation) | TeSO · AVS V1M | mIoU (%): 62.28 -> 66.02 | Improved |
| use frozen LLM with few-shot CoT prompts rather than a noun parser to collect text cues (ablation) | TeSO · AVS V1M | mIoU (%): 63.31 -> 66.02 | Improved |
| use few-shot instead of one-shot prompts (ablation) | TeSO · AVS V1M | mIoU (%): 63.87 -> 66.02 | Improved |
| use text-enhanced audio guidance instead of LLM filtering of predicted labels (ablation) | TeSO · AVS V1M | mIoU (%) drop under WGN-40dB: 32.8 -> 47.4 | Improved |
| use text-only LLaMA-2 reasoner instead of multimodal LLaVA-1.5 reasoner (ablation) | TeSO · AVS V1M | mIoU (%): 65.41 -> 66.02 | Improved |
| swap VGGish for stronger BEATs audio backbone (ablation) | TeSO · AVS V1M | mIoU (%): 66.02 -> 66.26 | No clear effect |
| use few-shot CoT prompt instead of zero-shot direct prompt (ablation) | TeSO text cue LLM · AVS V1M | mIoU (%): 63.83 -> 66.02 | Improved |
| TeSO versus prior methods on single-source V1S | TeSO · AVS V1S | mIoU (%): 82.68 -> 83.27 | No clear effect |
Closest papers by meaning
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.90)
- Dynamic Derivation and Elimination: Audio Visual Segmentation with Enhanced Audio Semantics (CVPR 2025 · similarity 0.90)
- Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation (CVPR 2024 · similarity 0.90)
- Improving Audio-Visual Segmentation with Bidirectional Generation (AAAI 2024 · similarity 0.89)
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.89)
- SAM2-LOVE: Segment Anything Model 2 in Language-aided Audio-Visual Scenes (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
