Research map / Image generation / Autoregressive and latent generation

Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusion

CVPR 2025 · Image generation › Autoregressive and latent generation · approach: Classical generative image synthesis · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Uni-Renderer jointly models rendering and inverse rendering as two cross-conditioned generation tasks in one dual-stream diffusion model (UniDiffuser-inspired), with a cycle-consistency loss, trained on rendered Objaverse material data.

Problem

Rendering (intrinsic attributes to RGB) and inverse rendering (RGB to intrinsic attributes such as metallic, roughness, albedo, normal, lighting) are typically handled separately; existing rendering is costly and inverse rendering is ambiguous.

Approach

Model rendering and inverse rendering as two conditional distributions in a single UniDiffuser-style diffusion model with distinct per-branch timesteps (one branch at t=0), a dual-stream module cross-conditioning two pretrained diffusion models, and a cycle-consistent re-rendering loss.

Key results

• Inverse rendering albedo PSNR 23.20 and LPIPS 0.0532, outperforming Intrinsic Image Diffusion (21.83 PSNR) and Yi et al. (20.93 PSNR).
• Rendering with metallic PSNR 30.72 vs. 28.09 for Subias et al. and 24.25 for InstructPix2Pix trained on the same data.
• Ablation: removing the unified framework drops albedo PSNR from 23.20 to 18.62; removing the cycle-consistency term drops it to 21.20.

Measured effects reported in the paper

Surprising findings

• RGB2X, a separate-model diffusion baseline, scores lower albedo PSNR (18.15) than several optimization-based or older methods, suggesting decoupled diffusion pipelines struggle with ambiguity.

Limitations

• Domain gap between synthetic Objaverse training data and real-world images limits generalization.
• Performance degrades on complex or unfamiliar objects and environments.

Experimental setup

Datasets: Objaverse LHQ-1024

Benchmarks: Objaverse held-out objects (100 unseen objects)

Metrics: PSNR SSIM LPIPS MSE cosine similarity

Baselines: Yi et al. IntrinsicAnything Intrinsic Image Decomposition (IntrinImgDecomp) Intrinsic Image Diffusion RGB2X NVdiffRec GaussianShader Wonder3D Subias et al. InstructPix2Pix

Builds on: UniDiffuser InstructPix2Pix Latent Diffusion (VAE latent)

Closest papers by meaning

• Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models (CVPR 2025 · similarity 0.93)
• Channel-wise Noise Scheduled Diffusion for Inverse Rendering in Indoor Scenes (CVPR 2025 · similarity 0.88)
• BLURD: Benchmarking and Learning using a Unified Rendering and Diffusion Model (NeurIPS 2024 · similarity 0.88)
• Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling (ICLR 2025 · similarity 0.87)
• UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesis (NeurIPS 2025 · similarity 0.87)
• Coherent 3D Scene Diffusion From a Single RGB Image (NeurIPS 2024 · similarity 0.87)