Research map / Architectures and efficiency / RNNs and normalization
The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations
Kushal Vyas, Alper Kayabasi, Daniel Kim, Vishwanath Saragadam, Ashok Veeraraghavan, Guha Balakrishnan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain INR on unstructured Uniform noise instead of real images (SNP: Uniform vs STRAINER) (ablation) | STRAINER-style SIREN encoder · CelebA-HQ image fitting | PSNR (dB): 57.8 -> 85.7 | Improved |
| pretrain on Gaussian noise instead of real images (ablation) | STRAINER-style SIREN encoder · CelebA-HQ image fitting | PSNR (dB): 57.8 -> 80.0 | Improved |
| pretrain on 1/f spectrum noise instead of real face images (ablation) | STRAINER-style SIREN encoder · CelebA-HQ image fitting | PSNR (dB): 57.8 -> 56.4 | No clear effect |
| pretrain on noise instead of real images, evaluated on OASIS-MRI (ablation) | STRAINER-style SIREN encoder · OASIS-MRI image fitting | PSNR (dB): STRAINER 62.80 -> SNP: Uniform 79.3 | Improved |
| use Dead Leaves noise pretraining instead of spectrum noise (ablation) | SNP · CelebA-HQ image fitting | PSNR (dB): 56.4 -> 55.0 | Hurt |
| add SNP pretraining to SIREN INR (any pretraining on noise) | SIREN · CelebA-HQ image fitting | PSNR (dB): 44.9 -> 56.4 (Spectrum) | Improved |
| use unstructured noise (Uniform, Gaussian) pretraining instead of SIREN random init (ablation) | INR denoising · CelebA-HQ denoising | PSNR (dB): SIREN 28.3 -> Uniform 23.0, Gaussian 23.8 | Hurt |
| use spectrum noise pretraining instead of SIREN random init for denoising | INR denoising · CelebA-HQ denoising | early-stopping iteration: 139 -> 78 | Improved |
| add SNP: Spectrum initialization to ResFields | ResFields · Pexels video fitting | mean PSNR (dB): 29.5 -> 31.1 | Improved |
| add SNP: Spectrum initialization to ResFields | ResFields · Pexels video denoising | mean PSNR (dB): 27.0 -> 28.0 | Improved |
| use Uniform instead of Spectrum noise for video (ablation) | ResFields + SNP · Pexels video fitting | mean PSNR (dB): 31.1 -> 30.9 | Hurt |
| choose Uniform noise over Spectrum noise pretraining (ablation) | SNP · fitting vs denoising tradeoff, CelebA-HQ | fitting PSNR vs denoising PSNR: fitting 56.4 -> 85.7; denoising 27.6 -> 23.0 | Mixed |
Closest papers by meaning
- VI^3NR: Variance Informed Initialization for Implicit Neural Representations (CVPR 2025 · similarity 0.90)
- Grids Often Outperform Implicit Neural Representation at Compressing Dense Signals (NeurIPS 2025 · similarity 0.89)
- Learning Transferable Features for Implicit Neural Representations (NeurIPS 2024 · similarity 0.88)
- Implicit Representations via Operator Learning (ICML 2024 · similarity 0.87)
- MIRE: Matched Implicit Neural Representations (CVPR 2025 · similarity 0.86)
- Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
