Research map / Image generation / Autoregressive and latent generation
MatSynth: A Modern PBR Materials Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MatSynth, a public dataset of 4,069 tileable 4K CC0/CC-BY PBR materials with augmented crops and renders, and shows it improves SVBRDF acquisition and MatFuse generation.
Problem
Material acquisition and generation research has relied on one small public dataset (Deschaintre et al. 2018) or on private company libraries, limiting progress and reproducibility.
Approach
A systematically gathered, larger, higher-resolution, permissively licensed set of tileable PBR materials with standardized maps, metadata, height-based blending, rotation/crop augmentation and multi-illumination renders, plus a two-pass rendering strategy that keeps pixel alignment while preserving specular highlights.
Key results
- Retraining Deschaintre et al. and SurfaceNet on MatSynth lowers rendering RMSE (e.g., SurfaceNet 0.161 to 0.135) and roughness RMSE (SurfaceNet 0.221 to 0.162).
- MatFuse FID drops from 239.9 (no fine-tuning) to 210.3 (fine-tuned) and 89.84 when trained from scratch on MatSynth.
- SSIM and LPIPS on renderings improve across methods; for SurfaceNet LPIPS falls from 0.395 to 0.281.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on MatSynth instead of the original dataset | Deschaintre et al. (2018) · Test set (89 materials), rendered materials | RMSE: 0.172 -> 0.160 | Helped |
| train on MatSynth instead of the original dataset | SurfaceNet · Test set (89 materials), rendered materials | RMSE: 0.161 -> 0.135 | Helped |
| train on MatSynth instead of the original dataset | Deschaintre et al. (2018) · Test set (89 materials) | Roughness RMSE: 0.322 -> 0.215 | Helped |
| train on MatSynth instead of the original dataset | SurfaceNet · Test set (89 materials) | Roughness RMSE: 0.221 -> 0.162 | Helped |
| train on MatSynth instead of the original dataset | Deschaintre et al. (2018) · Test set (89 materials) | Specular RMSE: 0.119 -> 0.080 | Helped |
| train on MatSynth instead of the original dataset | SurfaceNet · Test set (89 materials) | Specular RMSE: 0.086 -> 0.057 | Helped |
| train on MatSynth instead of the original dataset | Deschaintre et al. (2018) · Test set (89 materials), rendered materials | SSIM: 0.532 -> 0.560 | Helped |
| train on MatSynth instead of the original dataset | SurfaceNet · Test set (89 materials), rendered materials | SSIM: 0.494 -> 0.613 | Helped |
| train on MatSynth instead of the original dataset | Deschaintre et al. (2018) · Test set (89 materials), rendered materials | LPIPS: 0.560 -> 0.294 | Helped |
| train on MatSynth instead of the original dataset | SurfaceNet · Test set (89 materials), rendered materials | LPIPS: 0.395 -> 0.281 | Helped |
| fine-tune the diffusion model on MatSynth | MatFuse · Material generation | FID: 239.9 -> 210.3 | Helped |
| train the model on MatSynth from scratch | MatFuse · Material generation | FID: 239.9 -> 89.84 | Helped |
Limitations
- Gap remains between real material photographs and rendered synthetic materials.
- Some metadata (author, description, physical size) is available only for subsets of materials.
- Generation FID gains partly attributed to retrained autoencoder, confounding comparison.
Experimental setup
Datasets: MatSynth Deschaintre et al. 2018 material dataset
Benchmarks: MatSynth test set (89 materials)
Metrics: RMSE cosine distance (normals) SSIM LPIPS FID class distribution via CLIP zero-shot classification
Baselines: Deschaintre et al. 2018 SurfaceNet MatFuse
Builds on: MatFuse SurfaceNet Deschaintre et al. 2018 SVBRDF estimation ControlMat
Closest papers by meaning
- Material Palette: Extraction of Materials from a Single Image (CVPR 2024 · similarity 0.88)
- TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splatting (CVPR 2025 · similarity 0.86)
- Material Anything: Generating Materials for Any 3D Object via Diffusion (CVPR 2025 · similarity 0.86)
- Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials (NeurIPS 2024 · similarity 0.86)
- MaRI: Material Retrieval Integration across Domains (CVPR 2025 · similarity 0.84)
- VLMaterial: Procedural Material Generation with Large Vision-Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.