Research map / Video / Video generation
MACS: Multi-source Audio-to-image Generation with Contextual Significance and Semantic Alignment
Hao Zhou, Xiaobao Guo, Yuzhe Zhu, Adams Wai-Kin Kong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use spectrogram separator instead of MixIT waveform separator (ablation) | MACS · LLP-multi | FID: 98.73 -> 87.09 | Improved |
| use spectrogram separator instead of MixIT waveform separator (ablation) | MACS · LLP-multi | CLIP-FID: 28.42 -> 20.47 | Improved |
| use spectrogram separator instead of MixIT waveform separator (ablation) | MACS · LLP-multi | IIS: 0.5782 -> 0.6269 | Improved |
| add MACS multi-source separation to AudioToken (ablation) | AudioToken · LLP-multi | FID: 143.62 -> 130.77 | Improved |
| add MACS multi-source separation to AudioToken (ablation) | AudioToken · AudioSet-Eval | FID: 102.85 -> 96.93 | Improved |
| add MACS multi-source separation to AudioToken (ablation) | AudioToken · Landscape | FID: 236.63 -> 202.54 | Improved |
| add ranking loss (ablation) | MACS · LLP-multi | all metrics | Improved |
| add contrastive loss (ablation) | MACS · LLP-multi | all metrics | Improved |
| add decoupled cross-attention (ablation) | MACS · LLP-multi | all metrics | Improved |
| separate audio before generation (MACS vs mixed-audio conditioning) | MACS · LLP-multi | FID: 143.62 -> 87.09 | Improved |
Closest papers by meaning
- V2A-Mapper: A Lightweight Solution for Vision-to-Audio Generation by Connecting Foundation Models (AAAI 2024 · similarity 0.89)
- Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation (AAAI 2024 · similarity 0.89)
- From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation (ICML 2024 · similarity 0.88)
- Read, Watch and Scream! Sound Generation from Text and Video (AAAI 2025 · similarity 0.88)
- AudioCALM: Continuous Autoregressive Language Modeling for Universal Audio Generation (NeurIPS 2026 · similarity 0.87)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
