Research map / Probabilistic and scientific modeling / Dynamical systems
A sampling theory perspective on activations for implicit neural representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows via sampling theory/Riesz bases that sinc activations are optimal for INRs, validates on images, NeRF, and SINDy-based dynamical system discovery.
Problem
Activation functions in implicit neural representations (INRs) are studied piecemeal, lacking a unified theory of which activations are optimal for signal reconstruction and how they relate to dynamical systems modeling.
Approach
Views INR activations as generator functions of shift-invariant spaces; a Riesz basis plus partition-of-unity condition determines approximation ability, showing sinc is optimal among common activations, and uses sinc-INRs to denoise and differentiate data for SINDy.
Key results
- On NeRF real synthetic dataset, sinc-INR reaches 31.37 PSNR / 0.947 SSIM, versus 31.13 for Gaussian, 30.33 for wavelet and 28.96 for sinusoid.
- In SINDy Lorenz at noise n=1.0, sinc-based derivatives give 39.3 PSNR versus 20.3 for the baseline spectral derivative method.
- Image reconstruction results show sinc performs better or on-par with other activations across sampling rates after per-activation bandwidth grid search.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Sinc instead of Gaussian activation | SINDy reconstructions on Rossler · Rossler, n = 0.1 | PSNR: 46.6 -> 48.9 | Helped |
| Use Sinc instead of Gaussian activation | SINDy reconstructions on Rossler · Rossler, n = 0.5 | PSNR: 37.1 -> 42.8 | Helped |
| Use Sinc instead of Gaussian activation | SINDy reconstructions on Rossler · Rossler, n = 1. | PSNR: 33.6 -> 38.5 | Helped |
| Use Sinc instead of Gaussian activation | SINDy reconstructions on Lorenz · Lorenz, n = 0.1 | PSNR: 45.7 -> 46.2 | Helped |
| Use Sinc instead of Gaussian activation | SINDy reconstructions on Lorenz · Lorenz, n = 0.5 | PSNR: 39.1 -> 40.9 | Helped |
| Use Sinc instead of Gaussian activation | SINDy reconstructions on Lorenz · Lorenz, n = 1. | PSNR: 35.7 -> 39.3 | Helped |
| Use Sinc instead of Gaussian activation | INR novel view synthesis · Real synthetic dataset | PSNR: 31.13 -> 31.37 | Helped |
| Use Sinc instead of Gaussian activation | INR novel view synthesis · Real synthetic dataset | SSIM: 0.947 -> 0.947 | No clear effect |
Surprising findings
- Sinc-INRs with a single tuned bandwidth across an entire dataset outperform or match other activations, suggesting hyperparameter sensitivity is a key practical issue.
Limitations
- Optimality result is for Riesz-basis / shift-invariant space approximation and shallow networks; deep extension is sketched in appendix.
- Image and NeRF gains over Gaussian are marginal; practical advantage is small in several settings.
- Partition-of-unity condition does not hold for many activations; wavelet Riesz property not fully characterized.
Experimental setup
Datasets: DIV2K (sub-sampled set) NeRF real synthetic dataset Lorenz system simulated data Rossler system simulated data Vanderpol, Limit cycle, Duffing systems
Benchmarks: DIV2K subset image reconstruction NeRF real synthetic dataset SINDy Lorenz and Rossler reconstructions
Metrics: PSNR SSIM
Baselines: Gaussian activation Sinusoid activation Wavelet activation spectral derivatives (SINDy baseline)
Builds on: SIREN Fourier positional encoding (NeRF) Gaussian-activated INRs (GaussNet) wavelet-activated INRs (WIRE) SINDy
Closest papers by meaning
- Implicit Neural Representations and the Algebra of Complex Wavelets (ICLR 2024 · similarity 0.90)
- Grids Often Outperform Implicit Neural Representation at Compressing Dense Signals (NeurIPS 2025 · similarity 0.89)
- MIRE: Matched Implicit Neural Representations (CVPR 2025 · similarity 0.89)
- Implicit Representations via Operator Learning (ICML 2024 · similarity 0.88)
- FINER: Flexible Spectral-bias Tuning in Implicit NEural Representation by Variable-periodic Activation Functions (CVPR 2024 · similarity 0.87)
- FreSh: Frequency Shifting for Accelerated Neural Representation Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.