Research map / Architectures and efficiency / RNNs and normalization
Implicit Neural Representations and the Algebra of Complex Wavelets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives a time-frequency (Fourier convolution) analysis of wavelet-activated INRs (WIRE-style), shows complex/progressive wavelets form an algebra preserving band-pass support, and proposes a split scaling+Gabor INR with wavelet-modulus-maxima initialization.
Problem
Understanding the function class represented by implicit neural representations beyond the first layer, and how wavelet/complex activation templates shape their frequency support and singularity representation.
Approach
Shows that a WIRE INR's output Fourier transform is a sum of self-convolutions of the first-layer template atoms, so progressive (complex) wavelet templates keep Fourier support in conic sets; this motivates a split linear scaling INR plus nonlinear Gabor INR and initializing biases at wavelet modulus maxima / Canny edges.
Key results
- WMM/Canny-based initialization gives nearly 30 dB higher PSNR than uninitialized INR on images with isolated singularities (circular edge example).
- For 1D signals, strategically initialized INRs achieve roughly an order of magnitude lower MSE than random initialization across K in {1,3,5,7}.
- Theory: INR output Fourier transform is a finite sum of self-convolutions of first-layer template Fourier transforms, preserving conic/band-pass Fourier support.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize Gabor INR atoms at wavelet modulus maxima instead of random positions | Split scaling and Gabor INR · 1D target signal | MSE: approximately an order of magnitude less | Helped |
| Initialize Gabor INR using Canny edge locations and directions | Split scaling and Gabor INR · Images with isolated singularities | PSNR: nearly 30dB higher PSNR | Helped |
| Initialize Gabor INR using Canny edge locations and directions | Split scaling and Gabor INR · Images with dense texture, including chest X-ray | PSNR | No clear effect |
| Remove nonlinear activations after the Gabor template atoms (ablation) | Gabor INR · 1D target signal | singularity resolution | Hurt |
| Use nonlinear activations after the Gabor template atoms rather than a linear combination (ablation) | Gabor INR · 1D target signal | high-frequency feature resolution | Helped |
Surprising findings
- WMM-based initialization offers limited advantage for images with dense texture such as chest X-ray scans.
Limitations
- Theory is local and relies on polynomial activation approximations; analytic/continuous extension is only sketched.
- Progressive (complex) template functions incur large error when fitting real-valued signals unless the real part is taken.
- Empirical evaluation is small-scale (1D signal and few images).
Experimental setup
Datasets: 1D test signal natural and medical test images
Metrics: MSE PSNR
Baselines: SIREN WIRE random initialization
Builds on: WIRE SIREN Fourier features
Closest papers by meaning
- A sampling theory perspective on activations for implicit neural representations (ICML 2024 · similarity 0.90)
- Implicit Representations via Operator Learning (ICML 2024 · similarity 0.89)
- MIRE: Matched Implicit Neural Representations (CVPR 2025 · similarity 0.88)
- Grids Often Outperform Implicit Neural Representation at Compressing Dense Signals (NeurIPS 2025 · similarity 0.88)
- FINER: Flexible Spectral-bias Tuning in Implicit NEural Representation by Variable-periodic Activation Functions (CVPR 2024 · similarity 0.87)
- VI^3NR: Variance Informed Initialization for Implicit Neural Representations (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.