Research map / 3D vision / Face and shape
FreSh: Frequency Shifting for Accelerated Neural Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreSh, which picks INR embedding hyperparameters (e.g., Siren omega_0, Fourier sigma, Finer omega/k, Hashgrid resolution) by matching the DFT spectrum of an untrained model's output to the target signal's spectrum via Wasserstein distance, avoiding costly grid search.
Problem
INRs suffer spectral bias, and their embedding hyperparameters must be tuned per target signal; default values are suboptimal and grid search requires training many models.
Approach
The frequency spectrum of an untrained model's initial output correlates strongly with eventual performance, so selecting the embedding configuration whose initial output spectrum minimizes Wasserstein distance to the target's spectrum approximates grid-search results at a small fraction of the cost.
Key results
- On image fitting, Siren+FreSh averages 34.62 PSNR vs 33.85 for default Siren; Fourier+FreSh 33.45 vs 32.12.
- FreSh reaches comparable Siren quality to grid search at about 1/20 of the total grid-search time (30.3 h vs 584.2 h for image fitting).
- On NeRF synthetic scenes, Fourier+FreSh averages 32.63 PSNR vs 32.04 for the Fourier baseline, and Siren+FreSh 31.43 vs 31.21.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FreSh frequency alignment | Siren · Image representation (average) | PSNR: 33.85 -> 34.62 | Helped |
| Add FreSh frequency alignment | Fourier · Image representation (average) | PSNR: 32.12 -> 33.45 | Helped |
| Add FreSh frequency alignment | Finer · Image representation (average) | PSNR: 35.11 -> 35.03 | Hurt |
| Add FreSh frequency alignment | Finer with k=0 · Image representation (average) | PSNR: 34.81 -> 34.88 | Helped |
| Remove time from input coordinates and use FreSh | Siren video representation · Cat video | PSNR: 39.84 -> 40.61 | Helped |
| Add FreSh frequency alignment | Siren video representation with time as input · Cat video | PSNR: 38.72 -> 36.84 | Hurt |
| Add FreSh frequency alignment | Fourier video representation without time input · Bikes video | PSNR: 40.61 -> 41.13 | Helped |
| Add FreSh frequency alignment | Fourier · NeRF synthetic datasets (average) | PSNR: 32.04 -> 32.63 | Helped |
| Add FreSh frequency alignment | Finer · NeRF synthetic datasets (average) | PSNR: 31.64 -> 31.62 | No clear effect |
| Set spectrum size to 128 instead of 64 (ablation) | Siren + FreSh · Image representation (average) | PSNR: 34.59 -> 34.15 | Hurt |
| Set spectrum size to 128 instead of 64 (ablation) | Fourier + FreSh · Image representation (average) | PSNR: 33.47 -> 33.55 | Helped |
| Remove the Finer bias by setting k=0 (ablation) | Finer · Image representation (average) | PSNR: 35.11 -> 34.81 | Hurt |
Surprising findings
- For Hashgrid on NeRF, FreSh selects lower resolutions than the recommended range, improving quality, showing it can also reduce embedding frequency.
- Removing time as an input coordinate for video let FreSh improve results, suggesting spatial and temporal directions need different frequency magnitudes.
Limitations
- Does not account for direction-dependent frequency magnitudes, which matters for video.
- Cannot exceed the quality achievable by the underlying embedding under a full grid search, and is not applicable to NeRF or Wire with excessively high frequencies.
- Introduces a spectrum size hyperparameter n that may still need manual tuning.
Experimental setup
Datasets: FFHQ-1024 FFHQ-wild Wiki Art Chest X-Ray Kodak Cat video Bikes video NeRF synthetic
Benchmarks: FFHQ-1024 FFHQ-wild Wiki Art Chest X-Ray Kodak Cat video Bikes video NeRF synthetic
Metrics: PSNR SSIM training time
Baselines: Siren Fourier features Finer Finer k=0 Wire NeRF positional encoding Hashgrid grid search
Builds on: Siren Fourier features Finer Instant-NGP ResFields
Closest papers by meaning
- FINER: Flexible Spectral-bias Tuning in Implicit NEural Representation by Variable-periodic Activation Functions (CVPR 2024 · similarity 0.87)
- A sampling theory perspective on activations for implicit neural representations (ICML 2024 · similarity 0.87)
- MIRE: Matched Implicit Neural Representations (CVPR 2025 · similarity 0.87)
- VI^3NR: Variance Informed Initialization for Implicit Neural Representations (CVPR 2025 · similarity 0.87)
- Grids Often Outperform Implicit Neural Representation at Compressing Dense Signals (NeurIPS 2025 · similarity 0.87)
- Inductive Gradient Adjustment for Spectral Bias in Implicit Neural Representations (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.