Research map / Time series / Causal and generative time series
Neural Functions for Learning Periodic Signal
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeRT, an implicit neural representation that learns Fourier feature mappings and factorizes a signal into periodic and scale components for interpolation and extrapolation of periodic signals and time series.
Problem
Coordinate-based MLPs / implicit neural representations overfit and fail to extrapolate beyond the training region when the target signal is periodic, such as time series, ODE solutions and PDE solutions.
Approach
Encode coordinates with a learnable Fourier feature mapping and decode the signal as the product of a sinusoidal periodic factor (bounded in [-1,1]) and an unconstrained scale factor, trained end-to-end on the undecomposed signal, with a spatiotemporal coordinate system for multivariate series.
Key results
- On periodic time series, NeRT reports lower interpolation and extrapolation MSE than SIREN, FFN and WIRE in every listed configuration (e.g., Caiso 1-block interpolation 0.0047 vs 0.0182-0.0230 for baselines).
- On long-term ETTh1 with 50% observations dropped, NeRT achieves MSE 0.0911 versus 0.2173 (SIREN) and 0.3407 (FFN).
- On the 2D Helmholtz extrapolation task, only NeRT extrapolates accurately, reaching MSE 0.0009 at 10,000 epochs versus 0.2504 for SIREN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NeRT instead of the strongest INR baseline | periodic time-series INR modeling · Electricity interpolation and extrapolation, 1–3 missing blocks | MSE: 0.0200 -> 0.0061; 0.0256 -> 0.0057 | Helped |
| Use NeRT instead of the strongest INR baseline | periodic time-series INR modeling · Traffic interpolation and extrapolation, 1–3 missing blocks | MSE: 0.0140 -> 0.0057; 0.0113 -> 0.0050 | Helped |
| Use NeRT instead of the strongest INR baseline | periodic time-series INR modeling · Caiso interpolation and extrapolation, 1–3 missing blocks | MSE: 0.0179 -> 0.0047; 0.0466 -> 0.0131 | Helped |
| Use NeRT instead of the strongest listed baseline | long-term time-series imputation · ETTh1, 30% observations dropped | MSE: 0.1945 -> 0.0828 | Helped |
| Use NeRT instead of the strongest listed baseline | long-term time-series imputation · National Illness, 30% observations dropped | MSE: 0.1110 -> 0.0239 | Helped |
| Use NeRT instead of the strongest INR baseline | periodic-signal representation · 2D Helmholtz, 1,000 epochs | MSE: 0.2753 -> 0.0025 | Helped |
| Learn Fourier mapping parameters rather than fixing them at initialization (ablation) | NeRT · Electricity interpolation, 1 missing block | MSE: 0.0079 -> 0.0061 | Helped |
| Learn Fourier mapping parameters rather than fixing them at initialization (ablation) | NeRT · Electricity extrapolation, 1 missing block | MSE: 0.0069 -> 0.0057 | Helped |
| Learn Fourier mapping parameters rather than fixing them at initialization (ablation) | NeRT · NP interpolation and extrapolation, 1 missing block | MSE: 0.0190 -> 0.0149; 0.0313 -> 0.0235 | Helped |
| Use the proposed spatiotemporal coordinate mapping instead of a scaled timestamp alone (ablation) | NeRT and INR baselines · time-series representation and extrapolation | MSE | Helped |
Surprising findings
- Standard INRs (SIREN, FFN, WIRE) fail even on a noiseless damped harmonic oscillator and on sin(50x) extrapolation, despite fitting training data well.
- ReLU and Tanh MLPs struggle to even fit the training range on the Helmholtz problem, and error grows with more epochs for ReLU (3.71 to 20.26).
Limitations
- Vanilla INR trains one model per time series instance and does not emphasize recent observations; adaptation to new data (shift modulation) is left to future work.
- Extrapolation error grows for some settings (e.g., Traffic 3-block extrapolation 0.0115) and for undamped oscillator extrapolation (0.4109).
- Code is promised but not released at time of paper; the Appendix states it will be released online.
Experimental setup
Datasets: Electricity Traffic Caiso NP ETTh1 ETTh2 National Illness Minamitorishima atmospheric data harmonic oscillator ODE 2D Helmholtz equation
Benchmarks: Electricity Traffic Caiso NP ETTh1 National Illness 2D Helmholtz equation harmonic oscillator ODE atmospheric data
Metrics: MSE
Baselines: SIREN FFN WIRE ReLU MLP Tanh MLP Snake Transformer-based time series models NODE-based time series models
Builds on: SIREN Fourier feature networks WIRE neural tangent kernel Snake activation
Closest papers by meaning
- Rethinking Fourier Transform from A Basis Functions Perspective for Long-term Time Series Forecasting (NeurIPS 2024 · similarity 0.84)
- MPTSNet: Integrating Multiscale Periodic Local Patterns and Global Dependencies for Multivariate Time Series Classification (AAAI 2025 · similarity 0.84)
- FINER: Flexible Spectral-bias Tuning in Implicit NEural Representation by Variable-periodic Activation Functions (CVPR 2024 · similarity 0.84)
- Peri-midFormer: Periodic Pyramid Transformer for Time Series Analysis (NeurIPS 2024 · similarity 0.83)
- Spectral Transformer Neural Processes (NeurIPS 2026 · similarity 0.83)
- CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.