Research map / 3D vision / 3D generation
MET3R: Measuring Multi-View Consistency in Generated Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MEt3R, a pose-free, feed-forward metric for multi-view consistency of generated images that warps DINO+FeatUp features between DUSt3R-reconstructed views, and uses it to benchmark multi-view, video, and object-level generators plus an open multi-view latent diffusion model (MV-LDM).
Problem
Generated multi-view images lack paired ground truth, and existing metrics (FID, FVD, TSED, SED, NeRF-based scores) do not reliably measure 3D consistency between generated views, especially without camera poses.
Approach
Measure consistency by using DUSt3R point maps to unproject DINO features (upsampled with FeatUp) from one image into the other's view and computing cosine feature similarity in both directions, giving a pose-free, differentiable, gradual consistency score that is invariant to lighting and appearance.
Key results
- MV-LDM achieves MEt3R 0.036 vs GenWarp 0.120 and PhotoNVS 0.069 among multi-view methods, with FID 37.29, a favorable quality-consistency trade-off.
- DFM is most consistent among multi-view methods (MEt3R 0.026) but has the worst FID (73.02).
- Among video models, SVD achieves the best MEt3R (0.032), ahead of Ruyi-Mini-7B (0.047) and I2VGen-XL (0.050).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-view latent diffusion instead of single-view generation | RealEstate10K test generation · RealEstate10K test | MEt3R: 0.120 -> 0.036 | Helped |
| Use multi-view latent diffusion instead of single-view generation | RealEstate10K test generation · RealEstate10K test | FID: 29.80 -> 37.29 | Mixed |
| Use DFM instead of MV-LDM | RealEstate10K test generation · RealEstate10K test | MEt3R: 0.036 -> 0.026 | Helped |
| Use DFM instead of MV-LDM | RealEstate10K test generation · RealEstate10K test | FID: 37.29 -> 73.02 | Hurt |
| Use DINO feature cosine similarity instead of RGB PSNR/SSIM (ablation) | MEt3R · RealEstate10K test | consistency alignment | Helped |
| Use DINO features instead of DINOv2 features (ablation) | MEt3R · RealEstate10K test | MEt3R separation of model performance | Helped |
| Use DINO features instead of MaskCLIP features (ablation) | MEt3R · RealEstate10K test | MEt3R separation of model performance | Helped |
| Use SVD instead of I2VGen-XL | video generation · RealEstate10K test | MEt3R: 0.050 -> 0.032 | Helped |
Surprising findings
- PSNR and SSIM rate DFM's novel views higher than real videos, whereas MEt3R correctly ranks real videos as more consistent.
- DINOv2 and MaskCLIP features compress the metric range and reduce separation between inconsistent and consistent generations compared with DINO.
Limitations
- Metric depends on DUSt3R reconstruction quality, creating a non-zero lower bound even for real videos.
- DINO feature backbone choice affects separation; alternatives compress scores.
- Evaluation of video models is limited to 48 frames due to memory, and camera trajectories are not controlled across methods.
Experimental setup
Datasets: RealEstate10K Google Scanned Objects
Benchmarks: RealEstate10K Google Scanned Objects
Metrics: MEt3R TSED SED FVD FID FWS (PSNR)
Baselines: GenWarp PhotoNVS DFM SVD I2VGen-XL Ruyi-Mini-7B EpiDiff SyncDreamer VideoMV TSED SED FVD
Builds on: DUSt3R FeatUp DINO CAT3D Stable Diffusion 2.1
Closest papers by meaning
- HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D (CVPR 2024 · similarity 0.89)
- ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusion (CVPR 2024 · similarity 0.89)
- SyncDreamer: Generating Multiview-consistent Images from a Single-view Image (ICLR 2024 · similarity 0.88)
- MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds (CVPR 2025 · similarity 0.88)
- MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model (CVPR 2025 · similarity 0.88)
- Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.