Research map / Image generation / Fast diffusion sampling
Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the Mono2Stereo dataset (2.4M stereo pairs) and a new SIoU metric for stereo conversion, plus a dual-condition diffusion baseline trained with an Edge Consistency loss.
Problem
Stereo conversion (generating the right view from a left-view monocular image or video) lacks large-scale public training data, standardized benchmarks, and metrics that reflect stereoscopic effect; pixel metrics like RMSE/PSNR/SSIM fail to capture it.
Approach
Builds a large stereo dataset and proposes SIoU, an edge- and disparity-focused IoU metric that correlates with human judgments; the proposed model conditions a latent diffusion UNet on both the geometric (left-view) latent and the disparity-warped viewpoint latent, and adds an Edge Consistency loss on Sobel-filtered velocity predictions to counter identity-mapping degradation.
Key results
- SIoU reaches Spearman 0.84 correlation with human stereo judgments versus 0.21-0.26 for RMSE, PSNR and SSIM.
- Dual-condition model achieves SIoU 0.2836 on Mono2Stereo and 0.3398 on Inria 3DMovie, outperforming OWL3D (0.2781 and 0.2862).
- Edge Consistency loss raises the geometric-condition SIoU from 0.2418 to 0.2573 and the dual-condition model from 0.2533 to 0.2619 (ablation, Geo.+View. with EC loss).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use geometric and viewpoint conditions instead of geometric condition only (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2585 -> 0.2836 | Helped |
| use viewpoint condition instead of geometric condition only (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2585 -> 0.2809 | Helped |
| add depth maps to the geometric condition (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2533 -> 0.2551 | No clear effect |
| add edge maps to the geometric condition (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2533 -> 0.2543 | No clear effect |
| add depth maps to the viewpoint condition (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2604 -> 0.2605 | No clear effect |
| add edge maps to the viewpoint condition (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2604 -> 0.2596 | No clear effect |
| add occlusion masks to the viewpoint condition (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2604 -> 0.2581 | Hurt |
| add geometric condition alongside viewpoint condition (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2604 -> 0.2619 | Helped |
| add Edge Consistency loss to the geometric-condition model (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2418 -> 0.2533 | Helped |
| add Edge Consistency loss to the viewpoint-condition model (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2573 -> 0.2604 | Helped |
| add Edge Consistency loss to the dual-condition model (ablation) | Stereo conversion diffusion model · Mono2Stereo | SIoU: 0.2588 -> 0.2619 | Helped |
| use SIoU instead of RMSE, PSNR, or SSIM to rank stereo quality | Stereo quality evaluation · Human judgments of stereo quality | Spearman rank correlation: 0.26 -> 0.84 | Helped |
Surprising findings
- An identity mapping model (no stereo effect) scores competitively on RMSE, PSNR and SSIM, showing these metrics fail to capture stereo quality.
- Adding depth maps, edge maps, or occlusion masks as extra conditions generally did not help and sometimes hurt image quality metrics.
Limitations
- Image diffusion backbone rather than a video model, due to compute constraints.
- Weaker performance on complex and indoor scenes and on animation color consistency, per the per-scene analysis.
- SIoU depends on Canny edges and a binarization threshold, with a fixed alpha of 0.75.
Experimental setup
Datasets: Mono2Stereo Inria 3DMovie
Benchmarks: Mono2Stereo Inria 3DMovie
Metrics: SIoU RMSE PSNR SSIM Spearman rank correlation Kendall rank correlation
Baselines: 3D Photography RePaint StereoDiffusion OWL3D identity mapping
Builds on: Stable Diffusion v2 Marigold StereoDiffusion Depth Anything DDPM
Closest papers by meaning
- Any-Stereo: Arbitrary Scale Disparity Estimation for Iterative Stereo Matching (AAAI 2024 · similarity 0.88)
- MonSter: Marry Monodepth to Stereo Unleashes Power (CVPR 2025 · similarity 0.88)
- Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail (CVPR 2025 · similarity 0.87)
- PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching (NeurIPS 2025 · similarity 0.87)
- HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D (CVPR 2024 · similarity 0.86)
- Modeling Stereo-Confidence out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.