Research map / 3D vision / 3D generation

Thinking in Boxes: 3D Editing in Real Images Made Easy

NeurIPS 2026 · 3D vision › 3D generation · approach: Learning physics from video · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes "thinking in boxes": users specify source and target 3D boxes plus a depth-aligned checkerboard floor, and a FLUX-Kontext LoRA maps the box pair to a 3D edit of real photos, trained on synthetic Objaverse scenes and Objectron.

Problem

Text and 2D interfaces give weak, ambiguous control over large spatial transformations (object motion and camera changes) in real image editing, and prior 3D methods need per-image optimization, rely on depth-only signals, or fail on real photos.

Approach

Use input and output 3D boxes with color-coded faces as a structured specification of the edit, anchored by a depth-aligned checkerboard floor shared across object and camera motion, and feed the projected source/target layouts as extra tokens to a fine-tuned image editor.

Key results

• On the synthetic object-editing test set, the method reaches PSNR 23.686 and LPIPS 0.130, versus 20.852 and 0.226 for the best baseline FreeFine.
• On the Objectron camera-editing test set, mean distance drops to 29.406 versus 104.916 for SEVA (the best baseline on that metric).
• On real WildDet-3D object editing, the method has the lowest warp error (0.136) and mean distance (25.910), and ranks first or second on every metric.

Surprising findings

• Trained mostly on synthetic Blender renders plus only 10K real Objectron pairs, the model generalizes to real photos and to sketch-style line drawings despite training only on photographic data.
• Proprietary models Gemini and ChatGPT struggle to perform precise 3D edits, which the authors attribute to their text-based conditioning.

Limitations

• When objects have similar scales their boxes are indistinguishable, so the model outputs identity instead of committing to one transformation.
• Unintended pixel-level background alterations and artifacts are introduced during editing.
• Evaluated on a limited set of real images and user study; many comparisons rely on synthetic or self-constructed real test pairs.

Experimental setup

Datasets: Objaverse-XL Objectron WildDet-3D 3D-Fixer HDRIs and floor materials

Benchmarks: WildDet-3D object editing synthetic object editing test set synthetic camera editing test set Objectron camera editing test set

Metrics: PSNR SSIM LPIPS DreamSim DINO subject consistency DINO background consistency warp error mean distance IoU angular error

Baselines: SAM-3D 3D-Fixer SpatialEdit GeoDiffuser Diffusion Handles FreeFine SEVA Qwen-Camera-Control GBW

Builds on: FLUX.1 Kontext LoRA Objaverse-XL Objectron

Closest papers by meaning

• Image Sculpting: Precise Object Editing with 3D Geometry Control (CVPR 2024 · similarity 0.88)
• Towards Scalable and Consistent 3D Editing (ICML 2026 · similarity 0.88)
• Cameras as Relative Positional Encoding (NeurIPS 2025 · similarity 0.87)
• PrEditor3D: Fast and Precise 3D Shape Editing (CVPR 2025 · similarity 0.87)
• CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusion (CVPR 2025 · similarity 0.87)
• Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D (CVPR 2024 · similarity 0.85)