Research map / 3D vision / Face and shape
Representing Sounds as Neural Amplitude Fields: A Benchmark of Coordinate-MLPs and a Fourier Kolmogorov-Arnold Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds the first benchmark of 48 Coordinate-MLP configurations (3 positional encodings x 16 activations) for audio amplitude representation, then proposes Fourier-ASR, a Fourier-basis Kolmogorov-Arnold network with frequency-adaptive learning that represents audio without positional encoding.
Problem
Implicit neural representations of audio signals with Coordinate-MLPs are underexplored and sensitive to positional encoding, activation choice, and frequency-dependent initialization, limiting robustness.
Approach
Replace MLP nonlinearities with learnable Fourier-basis functions on network edges (Fourier-KAN), motivated by the Fourier series and Kolmogorov-Arnold theorems, so no positional encoding is needed; add a frequency-adaptive layer-wise frequency schedule and forward-propagation-based initialization (FaLS).
Key results
- Sine with NeFF positional encoding achieves 32.66 dB average SNR across the three benchmark clips, the best of the 48 configurations.
- On GTZAN, NeFF+Sine (MLP) reaches 18.04 dB average SNR versus 12.76 dB for Fourier-ASR (KAN), which uses no positional encoding.
- Fourier-ASR (KAN) outperforms Sine (MLP) by about 6 dB and B-Spline (KAN) by about 18 dB in SNR on the evaluated datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Fourier-ASR instead of Sine (MLP) | audio representation · GTZAN (Avg.) | SNR: Fourier-ASR(KAN) significantly outperforms Sine(MLP) (≈6dB ↑) | Helped |
| Use Fourier-ASR instead of B-Spline (KAN) | audio representation · GTZAN (Avg.) | SNR: Fourier-ASR(KAN) ... B-Spline(KAN) (≈18dB ↑) | Helped |
| Use NeFF+Sine instead of Fourier-ASR | audio representation · GTZAN and CSTR VCTK | SNR: 12.28dB ↑; 14.04dB ↑ | Mixed |
| Add positional encoding to Gaussian activation using RFF | Coordinate-MLP audio representation · Bach, Counting, and Blues | SNR: 11.02dB ↑ in SNR | Helped |
| Add positional encoding to Sine activation using Fourier features | Coordinate-MLP audio representation · Bach, Counting, and Blues | SNR: 18.96dB ↑ in SNR | Helped |
| Set ReLU+NeFF dimensionality to L=16 rather than L=32 (ablation) | Coordinate-MLP · Bach (7s) | SNR: 5.37 -> 24.76 | Helped |
| Use the inverted pyramid frequency setting [64, 5, 3] instead of [8, 8, 8] (ablation) | Fourier-KAN · audio representation | convergence | Helped |
| Use Sitzmann frequency-dependent initialization instead of Kaiming initialization (ablation) | Sine Coordinate-MLP · Bach | SNR: 2.5-e5 -> 13.36 | Helped |
Surprising findings
- Without frequency-dependent (Sitzmann) initialization, Sine-type networks fail to converge at all, with SNR diverging to extreme values under Kaiming or Xavier initialization.
- Positional encoding remains essential for Gaussian and Sine activations on audio, despite these activations being proposed to eliminate it for images and radiance fields.
Limitations
- Fourier-ASR (KAN) trails the best NeFF+Sine MLP configuration in SNR, attributed to existing optimization strategies not fitting KAN well.
- Evaluation is limited to fitting short clips (7s and 30s) and 20 files, without downstream compression or generation tasks.
Experimental setup
Datasets: GTZAN CSTR VCTK SIREN audio clips (Bach, Counting)
Benchmarks: GTZAN CSTR VCTK Bach Counting Blues
Metrics: SNR LSD
Baselines: Gaussian (MLP) Sine (MLP) B-Spline (KAN) ReLU Laplacian Incode-Sine Gabor wavelet
Builds on: SIREN NeRF Fourier feature networks Kolmogorov-Arnold Networks INCODE
Closest papers by meaning
- A sampling theory perspective on activations for implicit neural representations (ICML 2024 · similarity 0.86)
- ASMR: Activation-Sharing Multi-Resolution Coordinate Networks for Efficient Inference (ICLR 2024 · similarity 0.86)
- Learning Spatially Collaged Fourier Bases for Implicit Neural Representation (AAAI 2024 · similarity 0.85)
- FreSh: Frequency Shifting for Accelerated Neural Representation Learning (ICLR 2025 · similarity 0.85)
- Massive Sound Embedding Benchmark (MSEB) (NeurIPS 2025 · similarity 0.85)
- Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.