Research map / Multi-agent systems and games / LLM agents
NERFIFY: A Multi-Agent Framework for Turning NeRF Papers into Code
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Nerfify, a multi-agent framework that converts NeRF papers into trainable Nerfstudio plugins using grammar-constrained synthesis, citation recovery, and VLM-guided visual feedback, evaluated on a 30-paper benchmark.
Problem
Reimplementing NeRF papers is slow and generic paper-to-code systems usually produce non-trainable code for NeRF methods because of domain coupling and implicit citation dependencies.
Approach
Formalizes Nerfstudio's architecture as a context-free grammar to constrain LLM code synthesis, generates repositories in topological dependency order via a Graph-of-Thought multi-agent process, recovers borrowed components by traversing citation graphs, and refines outputs using PSNR-minima ROI analysis, cross-view checks, and VLM-guided patches.
Key results
- On never-implemented NeRF papers, Nerfify matches expert human PSNR within about 0.5 dB (e.g., KeyNeRF 26.12 vs 25.70 human), while Paper2Code, AutoP2C, GPT-5 and R1 fail to produce trainable code.
- Ablation: full Nerfify reaches semantic score 0.98 and 100% trainable, versus 0.45 score for one-shot generation without GoT.
- On Nerfstudio/original-repo comparisons, Nerfify reproduces Vanilla NeRF at 31.36 PSNR, matching the original repository.
Surprising findings
- Removing VLM feedback leaves the semantic score essentially unchanged (0.99 vs 0.98 full), and the paper notes it slightly lowers score by adjusting hyperparameters away from paper specs.
Limitations
- Evaluation on 30 papers with partial reliance on LLM-based scoring for novelty fidelity.
- Visual feedback sometimes deviates from paper-specified hyperparameters, trading fidelity for convergence.
- Comparisons on several benchmark results are incomplete in the main text (details deferred to supplementary).
Experimental setup
Datasets: Blender (Synthetic-NeRF) DTU
Benchmarks: Nerfify-Bench
Metrics: PSNR SSIM LPIPS executability novelty correct rate LLM-based semantic score
Baselines: Paper2Code AutoP2C GPT-5 R1 MetaGPT ChatDev DeepCode Expert implementation
Builds on: Graph of Thoughts Nerfstudio Paper2Code MinerU
Closest papers by meaning
- HyperFields: Towards Zero-Shot Generation of NeRFs from Text (ICML 2024 · similarity 0.87)
- LLaNA: Large Language and NeRF Assistant (NeurIPS 2024 · similarity 0.85)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.85)
- NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods (NeurIPS 2025 · similarity 0.84)
- InsertNeRF: Instilling Generalizability into NeRF with HyperNet Modules (ICLR 2024 · similarity 0.84)
- CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.