Research map / Vision-language and multimodal / Cross-modal learning
CMCGAN: A Uniform Framework for Cross-Modal Visual-Audio Mutual Generation
Wangli Hao, Zhaoxiang Zhang, He Guan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CMCGAN instead of S2IC to generate images from sound | S2IC · Sub-URMP, generated images | testing accuracy: 0.7556 -> 0.7661 | Improved |
| Use CMCGAN instead of I2S to generate sounds from images | I2S · Sub-URMP, generated sounds | testing accuracy: 0.1117 -> 0.5189 | Improved |
| Add generated image to sound input | Sub-URMP classification · Sub-URMP | classification accuracy: 0.9741 -> 0.9804 | Improved |
| Add generated sound to image input | Sub-URMP classification · Sub-URMP | classification accuracy: 0.9741 -> 0.9861 | Improved |
| Add cross-modal generation paths to the cycle model (ablation) | LCGAN · Sub-URMP cross-modal generation | generated modality quality | Improved |
| Remove latent vectors from the subnetworks (ablation) | CMCGAN · Sub-URMP cross-modal generation | generated modality quality | Hurt |
| Replace joint corresponding adversarial loss with standard adversarial loss (ablation) | CMCGAN · Sub-URMP generated images | image quality | Hurt |
| Use more convolutional and deconvolutional layers (ablation) | CMCGAN subnetworks · Sub-URMP cross-modal generation | generated modality quality | Improved |
Closest papers by meaning
- From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation (ICML 2024 · similarity 0.88)
- MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation (ICLR 2025 · similarity 0.87)
- CrossMAE: Cross-Modality Masked Autoencoders for Region-Aware Audio-Visual Pre-Training (CVPR 2024 · similarity 0.87)
- Asymmetric Hierarchical Anchoring for Robust Audio–Visual Cross-Modal Generalization (NeurIPS 2026 · similarity 0.87)
- X-AVDD: Cross-Attentive Audio-Visual Dataset Distillation (NeurIPS 2026 · similarity 0.86)
- Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
