AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Can Textual Semantics Mitigate Sounding Object Segmentation Preference?

Yaoting Wang, Peiwen Sun, Yuanchao Li, Honggang Zhang, Di Hu

ECCV 2024 · Vision-language and multimodal › Cross-modal learning · approach: Audio-visual source localization · 15 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add SeDAM module for text-audio fusion (ablation)TeSO · AVS V1MmIoU (%): 62.22 -> 66.02Improved
add PMQS module for prompting mask queries (ablation)TeSO · AVS V1MmIoU (%): 62.90 -> 66.02Improved
add dynamic mask based on attention score (ablation)TeSO · AVS V1MmIoU (%): 64.78 -> 66.02Improved
add InfoNCE latent component loss (ablation)TeSO · AVS V1MmIoU (%): 65.34 -> 66.02Improved
add text guidance in addition to audio (ablation)TeSO · AVS V1MmIoU (%): 62.28 -> 66.02Improved
use frozen LLM with few-shot CoT prompts rather than a noun parser to collect text cues (ablation)TeSO · AVS V1MmIoU (%): 63.31 -> 66.02Improved
use few-shot instead of one-shot prompts (ablation)TeSO · AVS V1MmIoU (%): 63.87 -> 66.02Improved
use text-enhanced audio guidance instead of LLM filtering of predicted labels (ablation)TeSO · AVS V1MmIoU (%) drop under WGN-40dB: 32.8 -> 47.4Improved
use text-only LLaMA-2 reasoner instead of multimodal LLaVA-1.5 reasoner (ablation)TeSO · AVS V1MmIoU (%): 65.41 -> 66.02Improved
swap VGGish for stronger BEATs audio backbone (ablation)TeSO · AVS V1MmIoU (%): 66.02 -> 66.26No clear effect
use few-shot CoT prompt instead of zero-shot direct prompt (ablation)TeSO text cue LLM · AVS V1MmIoU (%): 63.83 -> 66.02Improved
TeSO versus prior methods on single-source V1STeSO · AVS V1SmIoU (%): 82.68 -> 83.27No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy