Research map / Image generation / Text-to-image diffusion
Collaborative Control for Geometry-Conditioned PBR Image Generation
Shimon Vainer, Mark Boss, Mathias Parger, Konstantin Kutsy, Dante De Nigris, Ciara Rowles, Nicolas Perony, Simon Donné
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use bi-directional communication instead of one-way (ablation) | Collaborative Control · Objaverse held-out | CMMD (PBR): 16.44 -> 6.30 | Improved |
| use bi-directional instead of clockwise communication (ablation) | Collaborative Control · Objaverse held-out | CMMD (PBR): 6.78 -> 6.30 | Improved |
| use per-pixel MLP communication instead of single linear layer (ablation) | Collaborative Control · Objaverse held-out | CMMD (PBR): 6.30 -> 5.43 | No clear effect |
| use per-pixel linear communication instead of global attention (ablation) | Collaborative Control · Objaverse held-out | CMMD (PBR): 7.60 -> 6.30 | Improved |
| freeze RGB model with cross-network control instead of fine-tuning end-to-end (ablation) | PBR generation · OOD prompts (50 objects) | CLIPScore (Albedo): 25.04 -> 26.76 | Improved |
| freeze RGB model instead of fine-tuning, on in-distribution held-out data (ablation) | PBR generation · Objaverse held-out | FID (PBR): 11.41 -> 11.65 | No clear effect |
| train a dedicated PBR VAE instead of the RGB VAE on channel triplets (ablation) | Collaborative Control · Objaverse held-out | CMMD (PBR): 84.66 -> 6.30 | Improved |
| increase batch size 8x with 8 A100s (ablation) | Collaborative Control · Objaverse held-out | CMMD (PBR): 6.30 -> 2.96 | Improved |
| train at 256x256 instead of 512x512 (ablation) | Collaborative Control · Objaverse held-out | FID (PBR): 11.65 -> 9.82 | Mixed |
| disable text cross-attention in the PBR model (ablation) | Collaborative Control with 1% training data · Objaverse held-out | CMMD (PBR): 20.61 -> 6.25 | Improved |
| reduce training data from 98% to 1% (ablation) | Collaborative Control (no PBR prompt attention) · Objaverse held-out | CMMD (PBR): 6.30 -> 6.25 | No clear effect |
Closest papers by meaning
- MeshGen: Generating PBR Textured Mesh with Render-Enhanced Auto-Encoder and Generative Data Augmentation (CVPR 2025 · similarity 0.87)
- PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors (NeurIPS 2025 · similarity 0.86)
- IntrinsiX: High-Quality PBR Generation using Image Priors (NeurIPS 2025 · similarity 0.86)
- TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splatting (CVPR 2025 · similarity 0.85)
- Material Palette: Extraction of Materials from a Single Image (CVPR 2024 · similarity 0.83)
- DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
