Research map / 3D vision / 3D generation

MAGE : Single Image to Material-Aware 3D via the Multi-View G-Buffer Estimation Model

CVPR 2025 · 3D vision › 3D generation · approach: Diffusion-based novel view synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

MAGE predicts multi-view G-buffers (XYZ, normal, albedo, roughness, metallic) from a single image using a deterministic single-step network initialized from Zero123++, trained with image-space and physically based lighting response losses on a synthetic Objaverse-derived PBR dataset.

Problem

Single-image 3D reconstruction methods fail to recover physically based material properties (albedo, roughness, metallic) and often bake illumination into textures, limiting relighting in realistic scenarios.

Approach

Borrow the deferred-rendering G-buffer representation to decompose material-aware 3D generation into multi-view G-buffer estimation; convert a multi-view diffusion U-Net into a deterministic single-step predictor and supervise it with image-space losses including a split-sum-rendering-based lighting response loss that enforces physical consistency across G-buffer components.

Key results

• On G-buffer estimation, MAGE reaches albedo PSNR 22.243 and SSIM 0.8946 versus 20.489 and 0.8841 for the best prior baseline (RGB↔X)
• Normal estimation is comparable to GeoWizard (PSNR 28.399 vs 26.917 for IID+GeoWizard pipeline)
• Ablations show removing single-step design, image space loss, or lighting response loss lowers PSNR across all four attributes

Measured effects reported in the paper

Limitations

• Relies on synthetic training data with limited distribution
• Cannot handle complex materials such as anisotropic surfaces or subsurface scattering, nor refractive materials
• G-buffers estimated in 2D over only a few sparse viewpoints

Experimental setup

Datasets: Objaverse Freepik AI-generated and real captured test images

Benchmarks: Objaverse-based held-out G-buffer test set real captured and AI-generated wild images

Metrics: PSNR SSIM LPIPS

Baselines: MeshFormer RGB↔X IID GeoWizard multi-step diffusion G-buffer variant

Builds on: Zero123++ deferred rendering G-buffers split-sum image-based rendering single-step diffusion

Closest papers by meaning

• Material Palette: Extraction of Materials from a Single Image (CVPR 2024 · similarity 0.88)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference (NeurIPS 2025 · similarity 0.87)
• IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations (ICLR 2025 · similarity 0.86)
• One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (CVPR 2024 · similarity 0.86)
• LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields (CVPR 2025 · similarity 0.86)
• GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement (ICLR 2025 · similarity 0.86)