Research map / Vision-language and multimodal / Cross-modal learning
Audio-Visual Segmentation with Semantics
Jinxing Zhou, Xuyang Shen, Jianyuan Wang, Jiayi Zhang, Weixuan Sun, Jing Zhang, Stan T. Birchfield, Dan Guo, Lingpeng Kong, Meng Wang, Yiran Zhong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TPAVI audio-visual module (ablation) | AVS (ResNet50) · AVSBench S4 | mIoU (%): 70.12 -> 72.79 | Improved |
| add TPAVI audio-visual module (ablation) | AVS (PVT-v2) · AVSBench MS3 | mIoU (%): 48.21 -> 53.06 | Improved |
| add TPAVI audio-visual module (ablation) | AVS (PVT-v2) · AVSBench AVSS | mIoU (%): 27.71 -> 29.77 | Improved |
| add audio by direct addition of features (no TPAVI) (ablation) | AVS (PVT-v2) · AVSBench S4 | mIoU (%): 77.76 -> 77.65 | No clear effect |
| add audio by direct addition of features (no TPAVI) (ablation) | AVS (PVT-v2) · AVSBench MS3 | mIoU (%): 48.21 -> 51.55 | Improved |
| use TPAVI instead of direct audio-visual addition (ablation) | AVS (ResNet50) · AVSBench S4 | mIoU (%): 70.54 -> 72.79 | Improved |
| add L_AVM-AV regularization to BCE (ablation) | AVS (PVT-v2) · AVSBench MS3 | mIoU (%): 53.06 -> 54.00 | Improved |
| add L_AVM-AV regularization to BCE (ablation) | AVS (ResNet50) · AVSBench AVSS | mIoU (%): 18.88 -> 20.18 | Improved |
| add L_AVM-VV instead of L_AVM-AV (ablation) | AVS (PVT-v2) · AVSBench MS3 | mIoU (%): 53.06 -> 53.77 | Improved |
| apply TPAVI at all four stages instead of stage 4 only (ablation) | AVS (PVT-v2) · AVSBench MS3 | mIoU (%): 49.01 -> 54.00 | Improved |
| apply TPAVI at all four stages instead of best single stage (ablation) | AVS (ResNet50) · AVSBench AVSS | mIoU (%): 19.29 -> 20.18 | Improved |
| pretrain on Single-source subset (ablation) | AVS (ResNet50) · AVSBench MS3 | mIoU (%): 47.88 -> 54.33 | Improved |
Closest papers by meaning
- Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation (CVPR 2024 · similarity 0.94)
- Improving Audio-Visual Segmentation with Bidirectional Generation (AAAI 2024 · similarity 0.92)
- AVSegFormer: Audio-Visual Segmentation with Transformer (AAAI 2024 · similarity 0.92)
- Dynamic Derivation and Elimination: Audio Visual Segmentation with Enhanced Audio Semantics (CVPR 2025 · similarity 0.92)
- Audio-Visual Instance Segmentation (CVPR 2025 · similarity 0.91)
- Unsupervised Audio-Visual Segmentation with Modality Alignment (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
