Research map / 3D vision / Face and shape
Learning Spatially Collaged Fourier Bases for Implicit Neural Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCONE, an implicit neural representation that learns soft spatial masks to collage spatially-localized random Fourier feature patches, improving image fitting, video, and 3D shape reconstruction over SIREN, WIRE, MFN and Gaussian RFF baselines.
Problem
Global linear combinations of universal Fourier bases in implicit neural representations (INRs) force each frequency into regions where it is unnecessary, causing artifacts and limiting representation quality, especially locally.
Approach
Generate learnable, input-dependent soft spatial masks (via sin^2 activation) from previous-layer features and multiply them with per-layer random Fourier feature bases, so each basis is applied only to relevant regions; the masked patches are linearly mixed to form a collage.
Key results
- On Kodak image fitting, SCONE (sin^2) reaches mean 39.52 dB PSNR versus 37.41 dB for SIREN, a gain of over 3 dB.
- On video representation, SCONE achieves 27.41 dB PSNR versus 26.92 dB for SIREN and WIRE, with SSIM tied with SIREN at 0.70.
- On 3D shape representation, SCONE reaches 98.81% IoU and 0.0011 Chamfer Distance at about 400K parameters with 10K training steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use learnable soft spatial masks to collage Fourier bases instead of globally combining bases (ablation) | SCONE · Kodak image fitting | PSNR | Helped |
| Replace learnable soft masks with fixed random binary masks (ablation) | SCONE · Kodak image fitting | PSNR | Hurt |
| Use different mask activation functions (sigmoid, normalized tanh, or sin²) (ablation) | SCONE · Kodak image fitting | PSNR | No clear effect |
| Add a trainable input scaler before the normalized tanh mask activation (ablation) | SCONE · Kodak image fitting | PSNR | Helped |
| Use learnable spatial masks and collaged Fourier features rather than SIREN's global basis combination | SCONE · Kodak image fitting | PSNR | Helped |
| Use spatial collaging rather than global Fourier features | SCONE · Video representation | PSNR (dB): 26.92$\pm$0.87 -> 27.41 $\pm$ 0.83 | Helped |
| Use spatial collaging rather than global Fourier features | SCONE · Video representation | SSIM: 0.70$\pm$0.02 -> 0.70 $\pm$ 0.01 | No clear effect |
| Use spatially collaged Fourier features | SCONE · 3D shape representation (Stanford 3D scan) | IoU | Helped |
Surprising findings
- SIREN is a strong baseline at the ~200K parameter budget, outperforming WIRE and other baselines in image fitting.
- A random binary mask variant performs much worse than learned soft masks, indicating gains come from learned collaging rather than masking alone.
Limitations
- Baselines in 3D shape experiments were re-run under aligned settings and differ from previously reported values; WIRE and Gaussian RFF needed 20K steps.
- Ground-truth SDF labels for some 3D shapes contain artifacts from normal approximation.
- Results are on limited benchmarks (24 Kodak images, one video, Stanford scans).
Experimental setup
Datasets: Kodak Stanford 3D scan dataset Pexels cat video
Benchmarks: Kodak Stanford 3D scan Pexels cat video
Metrics: PSNR SSIM IoU Chamfer Distance
Baselines: SIREN Gaussian RFF MFN WIRE RBM
Builds on: SIREN Fourier feature networks WIRE multiplicative filter networks
Closest papers by meaning
- Implicit Neural Representations and the Algebra of Complex Wavelets (ICLR 2024 · similarity 0.87)
- Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion (NeurIPS 2024 · similarity 0.86)
- FINER: Flexible Spectral-bias Tuning in Implicit NEural Representation by Variable-periodic Activation Functions (CVPR 2024 · similarity 0.86)
- Grids Often Outperform Implicit Neural Representation at Compressing Dense Signals (NeurIPS 2025 · similarity 0.86)
- Sharpening Neural Implicit Functions with Frequency Consolidation Priors (AAAI 2025 · similarity 0.86)
- How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.