Research map / Image generation / Image restoration
CORE-MPI: Consistency Object Removal with Embedding MultiPlane Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CORE-MPI, which embeds a multiplane image into an RGB embedding image, removes objects with a single mask via a dual-branch LaMa inpainting network, and restores the MPI.
Problem
Object removal in multiplane image (MPI) novel view synthesis is hard: per-layer or per-view inpainting is slow and inconsistent, and MPI is high-dimensional and complex to edit.
Approach
Steganographically embed MPI (geometry and color) into a single RGB embedding image so that object removal needs only one inpainting pass; a dual-branch design (color restoration plus embedding inpainting) with a pseudo-reference loss and disparity consistency loss preserves color and geometry.
Key results
- On RealEstate10K, CORE-MPI reaches 29.790 PSNR and 0.0628 LPIPS in rendered views, the best among the compared methods (Layer-inpaint 28.699 PSNR).
- On UCSD, CORE-MPI reaches 38.367 PSNR and 43.829 FID, outperforming all baselines except on disparity L1.
- On the SPIn-NeRF dataset, CORE-MPI improves average LPIPS from 0.2551 to 0.1300 and FID from 184.90 to 110.74 without per-scene training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add disparity consistency loss to Embedding-Base (ablation) | CORE-MPI · RealEstate10K | disparity L1: 0.0680 -> 0.0467 | Helped |
| Add disparity consistency loss to Embedding-Base (ablation) | CORE-MPI · UCSD | disparity L1: 0.0196 -> 0.0159 | Helped |
| Add disparity consistency loss to Embedding-Base (ablation) | CORE-MPI · RealEstate10K | PSNR: 28.807 -> 28.821 | No clear effect |
| Add pseudo-reference loss to Embedding-Disparity (ablation) | CORE-MPI · RealEstate10K | PSNR: 28.821 -> 29.756 | Helped |
| Add pseudo-reference loss to Embedding-Disparity (ablation) | CORE-MPI · RealEstate10K | disparity L1: 0.0467 -> 0.0504 | Hurt |
| Add pseudo-reference loss to Embedding-Disparity (ablation) | CORE-MPI · UCSD | PSNR: 37.431 -> 38.310 | Helped |
| Add pseudo-reference loss to Embedding-Disparity (ablation) | CORE-MPI · UCSD | disparity L1: 0.0159 -> 0.0167 | Hurt |
| Add a color inpainting branch and fusion module to Embedding-Guide (ablation) | CORE-MPI · RealEstate10K | PSNR: 29.756 -> 29.790 | No clear effect |
| Add a color inpainting branch and fusion module to Embedding-Guide (ablation) | CORE-MPI · UCSD | PSNR: 38.310 -> 38.367 | No clear effect |
| Use CORE-MPI instead of Layer-inpaint | CORE-MPI · LLFF | PSNR: 34.256 -> 34.813 | Helped |
| Use CORE-MPI instead of SPIn-NeRF | object removal on the SPIn-NeRF dataset · SPIn-NeRF dataset, total | LPIPS: 0.2551 -> 0.1300 | Helped |
Surprising findings
- Pre-inpaint achieves the best disparity L1 error on UCSD but performs poorly on L2 error, indicating outliers.
Limitations
- The inpainting network does not explicitly model disparity; geometry is handled by a dual-network workaround.
- Embedding images are sensitive to distortion and manipulation, as noted for steganography generally.
Experimental setup
Datasets: RealEstate10K UCSD LLFF SPIn-NeRF dataset
Benchmarks: RealEstate10K UCSD LLFF SPIn-NeRF dataset
Metrics: PSNR SSIM LPIPS FID L1 disparity error L2 disparity error
Baselines: Pre-inpaint Post-inpaint Layer-inpaint Embedding-Base Embedding-Disparity Embedding-Guide SPIn-NeRF
Builds on: StereoMag LaMa multiplane image SPIn-NeRF
Closest papers by meaning
- In-N-Out: Lifting 2D Diffusion Prior for 3D Object Removal via Tuning-Free Latents Alignment (NeurIPS 2024 · similarity 0.83)
- PixPerfect: Seamless Latent Diffusion Local Editing with Discriminative Pixel-Space Refinement (NeurIPS 2025 · similarity 0.82)
- CLIPAway: Harmonizing focused embeddings for removing objects via diffusion models (NeurIPS 2024 · similarity 0.81)
- SCENIR: Visual Semantic Clarity through Unsupervised Scene Graph Retrieval (ICML 2025 · similarity 0.81)
- GeoRemover: Removing Objects and Their Causal Visual Artifacts (NeurIPS 2025 · similarity 0.81)
- Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.