Research map / Vision-language and multimodal / Cross-modal learning
Harmonic Canvas: Inversion-Free Editing for Visually-Guided Music Style Transfer
Yue Lei, Siqi Yang, Ting Zhong, Fan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual embeddings alongside text via cross-adapter (ablation) | Harmonic Canvas · Modality ablation | FAD: 3.15 -> 2.43 | Improved |
| use image embeddings only without text (ablation) | Harmonic Canvas · Modality ablation | FAD: 2.43 -> 4.06 | Hurt |
| use BLIP image captions instead of direct visual embeddings (ablation) | Harmonic Canvas · Modality ablation | FAD: 2.43 -> 3.49 | Hurt |
| add BLIP caption to text+image conditioning (ablation) | Harmonic Canvas · Modality ablation | FAD: 2.43 -> 2.86 | Hurt |
| add normalized chroma melodic guidance to FlowEdit-style inversion-free editing (ablation) | Harmonic Canvas · Inversion strategies | F0-PCC: 0.368 -> 0.416 | Improved |
| add normalized chroma melodic guidance (ablation) | Harmonic Canvas · Inversion strategies | CCS: 0.852 -> 0.878 | Improved |
| add normalized chroma melodic guidance (ablation) | Harmonic Canvas · Inversion strategies | FAD: 2.41 -> 2.43 | Hurt |
| use inversion-free FlowEdit instead of RF Inversion (ablation) | flow-based editing · Inversion strategies | F0-PCC: 0.323 -> 0.368 | Improved |
| increase chroma correction steps from 2 to 20 (ablation) | Harmonic Canvas · Sensitivity analysis | IMSM: 0.828 -> 0.815 | Hurt |
| increase chroma correction steps from 2 to 20 (ablation) | Harmonic Canvas · Sensitivity analysis | CCS: 0.878 -> 0.880 | No clear effect |
| increase guidance rate from 1e-4 to 5e-4 (ablation) | Harmonic Canvas · Sensitivity analysis | CCS: 0.878 -> 0.876 | No clear effect |
| use multimodal text and image cross-adapter over text-to-music baselines | Harmonic Canvas · User study | MOScon: 86.93 -> 89.20 | Improved |
Closest papers by meaning
- Music Style Transfer with Time-Varying Inversion of Diffusion Models (AAAI 2024 · similarity 0.91)
- MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models (CVPR 2024 · similarity 0.89)
- MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition (AAAI 2025 · similarity 0.87)
- MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners (ICML 2025 · similarity 0.85)
- V2Meow: Meowing to the Visual Beat via Video-to-Music Generation (AAAI 2024 · similarity 0.85)
- Deep Learning for Style Transfer and Experimentation with Audio Effects and Music Creation (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
