Research map / 3D vision / 3D generation
Perturb-and-Revise: Flexible 3D Editing with Generative Trajectories
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Perturb-and-Revise edits NeRFs with text prompts by interpolating source parameters toward random initialization (with the amount chosen from loss-landscape analysis), then refining with multi-view score distillation and an identity-preserving gradient.
Problem
Text-based 3D editing methods handle color, texture and style changes well but struggle with large geometric or appearance changes such as pose changes or adding objects.
Approach
Perturb NeRF parameters by linear interpolation with random initialization to escape local minima and follow generative trajectories, adaptively choose the perturbation level from the local loss decrease, and refine with an identity-preserving L1+perceptual gradient toward the source NeRF.
Key results
- On fashion editing, the method achieves averaged CLIP-Dir-Sim 0.0565 and LPIPS-alex 0.1034, a better trade-off than SDS (0.0438, 0.1533) and PDS (0.0285, 0.0215) and comparable to Instruct-NeRF2NeRF (0.0557, 0.1112).
- Identity-preserving refinement lowers LPIPS from 0.1147 to 0.1047 while CLIP-Dir-Sim drops from 0.0624 to 0.0565 and CLIP-Dir-Con rises from 0.7572 to 0.7642.
- Adaptive eta selection outperforms fixed eta values in averaged CLIP direction similarity and success rate while reducing cost relative to grid search.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Perturb-and-Revise instead of Instruct-NeRF2NeRF | Fashion object editing · Fashion object editing | CLIP-Dir-SimViT-B/32: 0.0583 -> 0.0594 | Helped |
| Use Perturb-and-Revise instead of Instruct-NeRF2NeRF | Fashion object editing · Fashion object editing | CLIP-Dir-SimViT-B/16: 0.0549 -> 0.0534 | Hurt |
| Use Perturb-and-Revise instead of Instruct-NeRF2NeRF | Fashion object editing · Fashion object editing | CLIP-Dir-SimViT-L/14: 0.0539 -> 0.0567 | Helped |
| Use Perturb-and-Revise instead of Instruct-NeRF2NeRF | Fashion object editing · Fashion object editing | CLIP-Dir-Sim averaged: 0.0557 -> 0.0565 | Helped |
| Use Perturb-and-Revise instead of Posterior Distillation | Fashion object editing · Fashion object editing | LPIPS VGG: 0.0337 -> 0.1060 | Hurt |
| Use Perturb-and-Revise instead of Posterior Distillation | Fashion object editing · Fashion object editing | LPIPS AlexNet: 0.0215 -> 0.1034 | Hurt |
| Add IPG refinement steps (ablation) | Perturb-and-Revise · Fashion object editing | CLIP-Dir-Sim: 0.0624 -> 0.0565 | Hurt |
| Add IPG refinement steps (ablation) | Perturb-and-Revise · Fashion object editing | CLIP-Dir-Con: 0.7572 -> 0.7642 | Helped |
| Add IPG refinement steps (ablation) | Perturb-and-Revise · Fashion object editing | LPIPS: 0.1147 -> 0.1047 | Helped |
| Use adaptive η selection instead of fixed η (ablation) | Perturb-and-Revise · Fashion object editing | CLIP direction similarity | Helped |
| Use adaptive η selection instead of fixed η (ablation) | Perturb-and-Revise · Fashion object editing | successful experiments | Helped |
Surprising findings
- Posterior Distillation achieves much lower LPIPS but mostly returns near-copies of the source, as shown by low CLIP directional similarity.
Limitations
- Limitations are deferred to the supplementary material and are not described in the main text.
- Evaluation relies on CLIP directional similarity and LPIPS, which do not directly measure geometric fidelity or realism.
Experimental setup
Datasets: Objaverse MVDream-generated fashion objects
Benchmarks: Fashion object editing set (150 examples) Objaverse general objects Real scene editing
Metrics: CLIP directional similarity CLIP directional consistency LPIPS
Baselines: Score Distillation Sampling Posterior Distillation Sampling Instruct-NeRF2NeRF Instruct-GS2GS MVDream regeneration
Builds on: Score Distillation Sampling Posterior Distillation Sampling Instruct-NeRF2NeRF MVDream
Closest papers by meaning
- ED-NeRF: Efficient Text-Guided Editing of 3D Scene With Latent Space NeRF (ICLR 2024 · similarity 0.89)
- DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservation (ICLR 2025 · similarity 0.87)
- PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code (ICLR 2024 · similarity 0.86)
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.86)
- h-Edit: Effective and Flexible Diffusion-Based Editing via Doob's h-Transform (CVPR 2025 · similarity 0.86)
- PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.