Research map / Architectures and efficiency / RNNs and normalization
How to Train Neural Field Representations: A Comprehensive Study and Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fit-a-NeF, a JAX library for fast parallel fitting of neural field datasets, uses it to study how NeF hyperparameters (shared initialization, overtraining, architecture expressivity) affect downstream classification, and releases the Neural Field Arena benchmark.
Problem
Neural fields used as data representations for downstream tasks are slow to fit at dataset scale, and how their hyperparameters affect downstream representation quality is poorly understood.
Approach
A JAX-based vmap/JIT parallelized library makes fitting millions of neural fields feasible, enabling a systematic study showing that shared initialization groups NeF representations in parameter space, that overtraining and overly expressive architectures hurt downstream accuracy even when reconstruction improves, and that off-grid/on-grid PSNR ratio is a useful heuristic.
Key results
- Shared initialization improves downstream test accuracy over random initialization: e.g., Neural MNIST 0.37 to 0.72 (+94.69%) and Neural ShapeNet 0.32 to 0.71 (+119.63%).
- Fit-a-NeF fits Neural MNIST in 17 seconds versus about 3 hours for a naive approach, a 100-1,300x speed-up; the full study took 62 A100 GPU days versus roughly 30 GPU years naively.
- On Neural MNIST, a simple GNN baseline reaches 96.40% test accuracy, above DWS (93.51%) and the Transformer (95.08%) reported on prior neural MNIST.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use shared rather than random initialization for fitting each neural field (ablation) | Neural field representations · Neural MNIST | test accuracy: 0.37±0.18 -> 0.72±0.17 | Helped |
| Use shared rather than random initialization for fitting each neural field (ablation) | Neural field representations · Neural CIFAR10 | test accuracy: 0.19±0.05 -> 0.27±0.08 | Helped |
| Use shared rather than random initialization for fitting each neural field (ablation) | Neural field representations · Neural MicroImageNet | test accuracy: 0.14±0.03 -> 0.20±0.05 | Helped |
| Use shared rather than random initialization for fitting each neural field (ablation) | Neural field representations · Neural ShapeNet | test accuracy: 0.32±0.18 -> 0.71±0.10 | Helped |
| Continue fitting after neural fields show signs of overtraining (ablation) | Neural field representations · Neural MNIST, Neural CIFAR10, Neural MicroImageNet, and Neural ShapeNet | test accuracy | Hurt |
| Increase neural-field hidden dimension (ablation) | Neural field representations · Neural datasets | test accuracy | Hurt |
| Increase neural-field hidden dimension (ablation) | Neural field representations · Neural datasets | PSNR | Helped |
Surprising findings
- Improved reconstruction quality does not monotonically improve representation quality; downstream accuracy peaks and then declines with overtraining.
- Increasing hidden dimension (more expressive NeF architectures) tends to lower representation quality even while PSNR rises.
Limitations
- Only images and shapes studied; NeRFs and videos excluded.
- Representation quality measured only via GNN classification accuracy, which may not transfer to other downstream tasks.
- Speedup depends on GPU memory and datapoint size.
Experimental setup
Datasets: MNIST CIFAR10 MicroImageNet (10-class ImageNet, 64x64) ShapeNetv2-10
Benchmarks: Neural MNIST Neural CIFAR10 Neural MicroImageNet Neural ShapeNet
Metrics: test accuracy PSNR IOU off-grid PSNR/PSNR ratio NMI speedup
Baselines: DWS Transformer (Kofinas et al.) Naive sequential NeF fitting
Builds on: SIREN Deep Weight Space networks Neural Field Transformer (Kofinas et al.) vmap Random Fourier features
Closest papers by meaning
- ResFields: Residual Neural Fields for Spatiotemporal Signals (ICLR 2024 · similarity 0.87)
- Neural Field Classifiers via Target Encoding and Classification Loss (ICLR 2024 · similarity 0.87)
- Learning Neural Exposure Fields for View Synthesis (NeurIPS 2025 · similarity 0.87)
- Grounding Continuous Representations in Geometry: Equivariant Neural Fields (ICLR 2025 · similarity 0.86)
- Neural Fields as Distributions: Signal Processing Beyond Euclidean Space (CVPR 2024 · similarity 0.86)
- From Activation to Initialization: Scaling Insights for Optimizing Neural Fields (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.