Research map / Image generation / Autoregressive and latent generation
MirrorVerse: Pushing Diffusion Models to Realistically Reflect the World
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds SynMirrorV2, an enhanced synthetic mirror-reflection dataset with random placement, rotation, grounding and multi-object pairing, and trains MirrorFusion 2.0 via a three-stage curriculum (synthetic single, synthetic multi, real MSD) to generate realistic mirror reflections by inpainting.
Problem
Diffusion models, including T2I models and inpainting methods, fail to generate geometrically accurate and photorealistic mirror reflections, especially across varied object poses, positions, multi-object occlusions, and real-world scenes.
Approach
Augment the synthetic rendering pipeline with random object positioning, random y-axis rotation, ground-plane grounding, and semantically paired multi-object scenes, then train a depth-conditioned dual-branch inpainting model (BrushNet-style) with a three-stage curriculum ending on real MSD data.
Key results
- On MirrorBenchV2 single-object split, MirrorFusion 2.0 (40k) reaches PSNR 18.79, SSIM 0.77, LPIPS 0.108 versus baseline MirrorFusion 18.31, 0.76, 0.122.
- In a user study, 84% of users preferred MirrorFusion 2.0 generations over the baseline.
- Fine-tuning on multiple objects improves multi-object split PSNR from 17.77 to 18.00 and LPIPS from 0.126 to 0.119.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train MirrorFusion 2.0 on SynMirrorV2 instead of using MirrorFusion | MirrorFusion · MirrorBenchV2 single-object | PSNR: 18.31 -> 18.79 | Helped |
| Train MirrorFusion 2.0 on SynMirrorV2 instead of using MirrorFusion | MirrorFusion · MirrorBenchV2 single-object | SSIM: 0.76 -> 0.77 | Helped |
| Train MirrorFusion 2.0 on SynMirrorV2 instead of using MirrorFusion | MirrorFusion · MirrorBenchV2 single-object | LPIPS: 0.122 -> 0.108 | Helped |
| Train MirrorFusion 2.0 on SynMirrorV2 instead of using MirrorFusion | MirrorFusion · MirrorBenchV2 single-object | CLIP similarity: 26.00 -> 25.96 | Hurt |
| Add fine-tuning on multiple-object data (ablation) | MirrorFusion 2.0 · MirrorBenchV2 multiple-object | PSNR: 17.77 -> 18.00 | Helped |
| Add fine-tuning on multiple-object data (ablation) | MirrorFusion 2.0 · MirrorBenchV2 multiple-object | SSIM: 0.743 -> 0.744 | Helped |
| Add fine-tuning on multiple-object data (ablation) | MirrorFusion 2.0 · MirrorBenchV2 multiple-object | LPIPS: 0.126 -> 0.119 | Helped |
| Add fine-tuning on multiple-object data (ablation) | MirrorFusion 2.0 · MirrorBenchV2 multiple-object | CLIP similarity: 26.17 -> 26.09 | Hurt |
| Use a single-branch SD Inpainting architecture with depth conditioning instead of the dual-branch architecture (ablation) | MirrorFusion 2.0 · Mirror reflection generation | reflection quality | Hurt |
| Fine-tune on real-world MSD data (ablation) | MirrorFusion 2.0 · MSD real-world held-out scenes | reflection quality | Helped |
| Add multiple-object examples to training (ablation) | MirrorFusion 2.0 · MirrorBenchV2 multiple-object | reflection quality | Helped |
Surprising findings
- Current T2I models (SD3.5, FLUX) fail to produce consistent geometrically accurate mirror reflections even when explicitly prompted.
Limitations
- Struggles with complex real-world scenes such as cluttered cables and reflections across multiple mirrors.
- Minor artifacts remain in scenes with more than two objects.
- Real-world generalization depends on fine-tuning with MSD data and a monocular depth estimator.
Experimental setup
Datasets: SynMirrorV2 Objaverse Amazon Berkeley Objects (ABO) MSD CC-textures PolyHaven HDRIs OBJECT 3DIT
Benchmarks: MirrorBenchV2 MSD GSO
Metrics: PSNR SSIM LPIPS CLIP similarity user preference
Baselines: MirrorFusion HD-Painter SDI+Depth Stable Diffusion 3.5 FLUX
Builds on: MirrorFusion BrushNet Stable Diffusion Inpainting ControlNet-style zero convolution
Closest papers by meaning
- MIRROR: Make Your Object-Level Multi-View Generation More Consistent with Training-Free Rectification (ICML 2025 · similarity 0.89)
- Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D (CVPR 2024 · similarity 0.89)
- PS-Diffusion: Photorealistic Subject-Driven Image Editing with Disentangled Control and Attention (CVPR 2025 · similarity 0.88)
- Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models (CVPR 2025 · similarity 0.87)
- Novel View Synthesis with Pixel-Space Diffusion Models (CVPR 2025 · similarity 0.87)
- Denoising Diffusion via Image-Based Rendering (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.