Research map / Vision-language and multimodal / Cross-modal learning
Multimodal Variational Auto-encoder based Audio-Visual Segmentation
Yuxin Mao, Jing Zhang, Mochu Xiang, Yiran Zhong, Yuchao Dai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use audio in a multimodal VAE rather than video-only CVAE (ablation) | ECMVAE · AVSBench MS3 | mIoU: 49.26 -> 54.99 | Improved |
| add latent space factorization into shared and specific codes (ablation) | ECMVAE · AVSBench MS3 | mIoU: 54.99 -> 56.38 | Improved |
| double the latent dimension without factorization (ablation) | ECMVAE · AVSBench MS3 | mIoU: 54.99 -> 55.09 | No clear effect |
| factorize in VAE latent space rather than AE feature space (ablation) | ECMVAE · AVSBench MS3 | mIoU: 54.82 -> 56.38 | Improved |
| use JS divergence instead of KL, PoE or MoE (ablation) | ECMVAE · AVSBench MS3 | mIoU: 57.53 -> 57.84 | Improved |
| add orthogonality constraint L_diff (ablation) | ECMVAE · AVSBench MS3 | mIoU: 57.51 -> 58.15 | Improved |
| add mutual information maximization L_sic (ablation) | ECMVAE · AVSBench MS3 | mIoU: 57.51 -> 57.84 | Improved |
| pre-train on S4 subset | ECMVAE (PVT) · AVSBench MS3 | mIoU: 57.84 -> 60.81 | Improved |
| pre-train on S4 subset | ECMVAE (R50) · AVSBench MS3 | mIoU: 48.69 -> 57.56 | Improved |
Closest papers by meaning
- Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation (CVPR 2024 · similarity 0.90)
- Improving Audio-Visual Segmentation with Bidirectional Generation (AAAI 2024 · similarity 0.89)
- Unsupervised Audio-Visual Segmentation with Modality Alignment (AAAI 2025 · similarity 0.88)
- AVSegFormer: Audio-Visual Segmentation with Transformer (AAAI 2024 · similarity 0.88)
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.88)
- Few-Shot Incremental Learning via Foreground Aggregation and Knowledge Transfer for Audio-Visual Semantic Segmentation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
