Research map / Image generation / Autoregressive and latent generation

MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attention

CVPR 2025 · Image generation › Autoregressive and latent generation · approach: Compositional scene generation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MEAT, a megapixel (1024x1024) multiview diffusion model for human novel-view generation that uses mesh attention over a monocular clothed-human mesh, plus keypoint conditioning and DNA-Rendering motion videos as training data.

Problem

Multiview diffusion models for human image-to-multiview generation fail to produce detailed, consistent results because dense cross-view attention does not scale to megapixel resolutions, and high-quality multiview human scan data is scarce.

Approach

Mesh attention uses rasterization and projection of a coarse clothed-human mesh to find direct cross-view pixel correspondences, so each pixel attends only to a small set of matched pixels in other views, reducing attention complexity; combined with keypoint conditioning, multi-scale VAE reference features, and a linear noise schedule, and training on multiview human motion videos.

Key results

• At 1024 resolution, MEAT reaches P-FID 10.60 and LPIPS 0.0751 versus 59.79 P-FID and 0.1456 LPIPS for Wonder3D (inference checkpoint, 256 resolution).
• Wonder3D retrained with keypoints achieves the best PPLC among methods in Table 2 (0.0895), with MEAT close behind (0.0928 at 1024 in the ablation table).
• Mesh attention at 1024 resolution uses 68/52 GB train/inference memory versus 186/471 GB for dense multiview attention, and is the only one trainable at 1024 among those compared.

Measured effects reported in the paper

Surprising findings

• Directly raising Stable Zero123 to 1024 resolution without a noise schedule change causes many artifacts, so schedule adjustment was needed.
• Keypoint conditioning substantially improved the numerical metrics of the retrained baselines, though they still trailed MEAT in (P-)FID and LPIPS.

Limitations

• Relies on a monocular-reconstructed clothed mesh (ECON/PIFuHD) as the central geometry, so mesh errors and keypoint errors affect results; perturbed keypoints lower metrics.
• Training data is limited to DNA-Rendering (real captured videos), with keypoint and mesh adaptation pipelines needed; generalization evaluated on the same dataset family.
• PPLC consistency metric is best for Wonder3D, which the authors attribute to duplicated back views with limited perspective variation.

Experimental setup

Datasets: DNA-Rendering THUman2.1

Benchmarks: DNA-Rendering Part 1

Metrics: PSNR SSIM LPIPS FID Patch-FID PPLC

Baselines: Stable Zero123 SyncDreamer Wonder3D SV3D MagicMan Dense multiview attention Epipolar attention

Builds on: Zero-1-to-3 Zero123++ Era3D EpiDiff ECON

Closest papers by meaning

• Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention (NeurIPS 2024 · similarity 0.90)
• Vivid-ZOO: Multi-View Video Generation with Diffusion Model (NeurIPS 2024 · similarity 0.89)
• MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model (CVPR 2025 · similarity 0.88)
• MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement (AAAI 2025 · similarity 0.88)
• HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation (CVPR 2024 · similarity 0.88)
• PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.88)