Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Differentiable Modal Synthesis for Physical Modeling of Planar String Sound and Motion Simulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DMSP, a differentiable modal-synthesis plus spectral-modeling neural network that simulates plucked nonlinear (planar) string motion and sound from physical parameters, trained on FDTD-simulated data.
Problem
Simulating spatio-temporal motion and sound of nonlinear planar strings guided by physical laws is underexplored; existing neural/differentiable synthesizers lack physical controllability and motion accuracy, while FDTD is costly and not differentiable.
Approach
A neural network that estimates modal frequencies and amplitudes from initial conditions and physical parameters, then applies learned AM and FM modulation to the linear modal solution and synthesizes the output with a harmonic-plus-noise pipeline, yielding a differentiable approximation of the nonlinear string PDE solution queryable at any (x,t).
Key results
- On nonlinear strings, DMSP-Hybrid reaches 15.67 dB SI-SDR and 4.77 dB MSS versus 17.25-ish baseline Modal at -16.61 dB SI-SDR, a nearly 31 dB SI-SDR gain over Modal.
- DMSP (fully learned modes) reaches -10.32 dB SI-SDR and 1.44 Hz pitch error on nonlinear strings, beating DDSPish (-25.95 dB SI-SDR, 3.31 Hz pitch).
- On linear strings, DDSPish-XFM and DDSPish achieve the best MSS, while DMSP-Hybrid attains the best linear SI-SDR (-2.84 dB) and SDR (1.50 dB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add AM and FM modulation to precomputed modal frequencies | DMSP-Hybrid vs Modal · Nonlinear strings (α>1) | SI-SDR: --16.611 -> 15.670 | Helped |
| Add AM and FM modulation to precomputed modal frequencies | DMSP-Hybrid vs Modal · Nonlinear strings (α>1) | MSS: 17.254 -> 4.772 | Helped |
| Estimate modes with a neural network instead of precomputing them | DMSP vs Modal · Nonlinear strings (α>1) | SI-SDR: --16.611 -> --10.315 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP-Hybrid · Nonlinear strings (α>1) | SI-SDR: --5.418 -> 15.670 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP-Hybrid · Nonlinear strings (α>1) | SDR: 1.509 -> 16.455 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP-Hybrid · Nonlinear strings (α>1) | MSS: 8.983 -> 4.772 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP-Hybrid · Nonlinear strings (α>1) | Pitch difference: 2.653 -> 1.027 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP · Nonlinear strings (α>1) | SI-SDR: --16.435 -> --10.315 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP · Nonlinear strings (α>1) | SDR: --1.074 -> 0.221 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP · Nonlinear strings (α>1) | MSS: 9.060 -> 5.656 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP · Nonlinear strings (α>1) | Pitch difference: 2.922 -> 1.437 | Helped |
| Add the pitch loss L_f0 (ablation) | DMSP-Hybrid · Linear strings (α=1) | SI-SDR: --2.919 -> --2.844 | No clear effect |
Surprising findings
- Removing the pitch loss L_f0 substantially degrades nonlinear performance across all metrics, e.g. DMSP-Hybrid SI-SDR drops from 15.67 to -5.42 dB.
- Modal synthesis with oracle mode frequencies gets the lowest linear pitch error but poor amplitude decay modeling, so its linear SI-SDR is weak relative to neural models.
Limitations
- Generalization to real-world measured physical parameters and sounds is not demonstrated.
- Performance on linear strings lags on MSS relative to DDSPish models; DMSP is weaker on linear SI-SDR than DMSP-Hybrid.
- Trained only on FDTD-simulated data with uniformly sampled parameters, with no real recordings.
Experimental setup
Datasets: StringFDTD-Torch simulated nonlinear planar string data
Benchmarks: StringFDTD-Torch simulated test set (715 strings, linear and nonlinear splits)
Metrics: SI-SDR SDR multi-scale spectral distance pitch difference (Hz)
Baselines: Modal synthesis DDSPish DDSPish-XFM
Builds on: DDSP modal synthesis StringFDTD-Torch CREPE
Closest papers by meaning
- Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion (ICML 2024 · similarity 0.82)
- MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence (NeurIPS 2024 · similarity 0.81)
- MIDI-GPT: A Controllable Generative Model for Computer-Assisted Multitrack Music Composition (AAAI 2025 · similarity 0.80)
- MuPT: A Generative Symbolic Music Pretrained Transformer (ICLR 2025 · similarity 0.80)
- Efficient Fine-Grained Guidance for Diffusion Model Based Symbolic Music Generation (ICML 2025 · similarity 0.80)
- MID-FiLD: MIDI Dataset for Fine-Level Dynamics (AAAI 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.