Research map / Vision-language and multimodal / Cross-modal learning
CPM: Class-conditional Prompting Machine for Audio-visual Segmentation
Yuanhong Chen, Chong Wang, Yuyuan Liu, Hu Wang, Gustavo Carneiro
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace softmax classifier with class-conditional GMM distribution modelling (CCDM) (ablation) | AVSegFormer* baseline · AVSBench-Semantics | mIoU: 53.04 -> 54.12 | Improved |
| add visual conditional prompting (VCP) (ablation) | baseline + CCDM · AVSBench-Semantics | mIoU: 54.12 -> 55.79 | Improved |
| add audio conditional prompting (ACP) (ablation) | baseline + CCDM · AVSBench-Semantics | mIoU: 54.12 -> 55.07 | Improved |
| add both ACP and VCP (ablation) | baseline + CCDM · AVSBench-Semantics | mIoU: 54.12 -> 56.41 | Improved |
| add prompting-based contrastive learning (PCL) (ablation) | CCDM + VCP + ACP · AVSBench-Semantics | mIoU: 56.41 -> 57.25 | Improved |
| use GMM distribution representation instead of point representation (linear softmax) (ablation) | CPM · AVSBench-Semantics | mIoU: +1.29% mIoU, +1.96% F-beta | Improved |
| use 3 GMM components rather than other counts (M=1 to 7) (ablation) | CPM · AVSBench-Semantics | mIoU: +1.42% mIoU, +1.95% F-beta (maximum improvement) | Improved |
| replace masked average pooling with masked max pooling in PCL (ablation) | CPM · AVSBench-Semantics | F-beta: -1.07% | Hurt |
| vary loss weight lambda (0.1 too small, 1.0 too large, 0.5 used) (ablation) | CPM · AVSBench-Semantics | mIoU | Mixed |
| apply CPM to stereo audio on VPO MSMI | CPM vs CAVP · VPO (MSMI) | mIoU: 60.36 -> 60.55 | No clear effect |
| use CPM instead of CAVP at original resolution | AVSS · AVSS (full, original resolution) | mIoU: 50.75 -> 57.25 | Improved |
| use CPM instead of CAVP for efficiency | AVSS · AVSS (full, original resolution) | inference speed: 20 fps -> 14 fps; 22 h -> 30 h training | Hurt |
Closest papers by meaning
- Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation (CVPR 2024 · similarity 0.91)
- AVSegFormer: Audio-Visual Segmentation with Transformer (AAAI 2024 · similarity 0.89)
- Revisiting Audio-Visual Segmentation with Vision-Centric Transformer (CVPR 2025 · similarity 0.88)
- Improving Audio-Visual Segmentation with Bidirectional Generation (AAAI 2024 · similarity 0.88)
- Unsupervised Audio-Visual Segmentation with Modality Alignment (AAAI 2025 · similarity 0.87)
- Think Before You Segment: An Object-aware Reasoning Agent for Referring Audio-Visual Segmentation (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
