Research map / Continual learning / Online and replay-based
Meta-Continual Learning of Neural Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces meta-continual learning of neural fields (MCL-NF), combining a modular, shared-initialization architecture with MAML-style meta-learning and a Fisher-information-weighted sample loss (FIM-NeRF) to reduce forgetting and speed convergence across image, audio, video and city-scale NeRF tasks.
Problem
Continual learning of neural fields suffers from catastrophic forgetting and slow convergence (many iterations per task), and existing meta-learning approaches for neural fields do not handle sequential data streams with limited memory.
Approach
Combine modular sub-networks with a shared meta-learned initialization (so module count can grow independently of task count) and reweight samples by a Fisher-information-based weight (FIM loss) instead of replaying past data, which the authors connect to mutual information maximization and provide convergence and generalization theorems for.
Key results
- On CelebA, Ours reaches 45.40 PSNR at step 4096 versus 45.28 for MOL and 28.06 for MAML+CL at step 4096.
- On LibriSpeech1 audio, Ours reaches 67.58 PSNR at step 4096 versus 55.76 for MOL and 41.62 for OML.
- On MatrixCity-B5, Ours (MIM) reaches 32.804 PSNR at step 1024 versus 26.5 for OML and 25.792 for MAML+CL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MCL-NF rather than the strongest listed baseline | Ours · CelebA | PSNR at Step 4096: 45.28 -> 45.40 | Helped |
| Use MCL-NF rather than the strongest listed baseline | Ours · ImageNette | PSNR at Step 4096: 45.74 -> 44.83 | Hurt |
| Use MCL-NF rather than the strongest listed baseline | Ours · FFHQ | PSNR at Step 4096: 35.30 -> 35.89 | Helped |
| Use MCL-NF rather than the strongest listed baseline | Ours · LibriSpeech1 | PSNR at Step 4096: 55.76 -> 67.58 | Helped |
| Use MCL-NF rather than the strongest listed baseline | Ours · LibriSpeech3 | PSNR at Step 4096: 41.97 -> 51.09 | Helped |
| Use MCL-NF rather than the strongest listed baseline | Ours · VoxCeleb2 | PSNR at Step 4096: 41.75 -> 47.80 | Helped |
| Use MCL-NF with MIM rather than MOL | Ours (MIM) · MatrixCity-B5 | PSNR at Step 1024: 30.569 -> 32.804 | Helped |
| Use MCL-NF with MIM rather than MOL | Ours (MIM) · MatrixCity-B6 | PSNR at Step 1024: 26.837 -> 30.761 | Helped |
| Add MIM to modularization (ablation) | Ours on MatrixCity-B5 · MatrixCity-B5 | PSNR at Step 1024: 32.712 -> 32.804 | Helped |
| Add MIM to modularization (ablation) | Ours on MatrixCity-B6 · MatrixCity-B6 | PSNR at Step 1024: 30.407 -> 30.761 | Helped |
| Add MIM at hidden dimension 256 (ablation) | Ours · MatrixCity-B5 | PSNR at Step 512: 32.088 -> 32.437 | Helped |
| Increase hidden dimension from 256 to 512 without MIM (ablation) | Ours (mod) · MatrixCity-B5 | PSNR at Step 512: 32.088 -> 32.486 | Helped |
Surprising findings
- Ours underperforms baselines at Step 1 on several image benchmarks (e.g., CelebA 24.22 vs 33.12 for MAML+CL in the resolution-reduced table), indicating weaker initial performance.
- Replay-based ER attains higher best PSNR than the proposed method on ImageNette (45.74 vs 44.83) in the main table.
Limitations
- Potential performance degradation over very long, dissimilar task sequences.
- Memory usage increases with the number of tasks.
- Possible limitations under abrupt, significant data distribution changes.
Experimental setup
Datasets: CelebA FFHQ ImageNette VoxCeleb2 LibriSpeech MatrixCity
Benchmarks: CelebA FFHQ ImageNette VoxCeleb2 LibriSpeech1 LibriSpeech3 MatrixCity-B5 MatrixCity-B6
Metrics: PSNR SSIM PESQ
Baselines: LearnedInit (MOL) OML MAML+CL EWC ER MER CL
Builds on: MAML EWC Online meta-learning (OML) MegaNeRF SwitchNeRF LearnedInit
Closest papers by meaning
- Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation (ICLR 2024 · similarity 0.87)
- Spatial Annealing for Efficient Few-shot Neural Rendering (AAAI 2025 · similarity 0.86)
- CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning (AAAI 2024 · similarity 0.86)
- Ced-NeRF: A Compact and Efficient Method for Dynamic Neural Radiance Fields (AAAI 2024 · similarity 0.86)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.86)
- FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.