Research map / 3D vision / Face and shape
QUADify: Extracting Meshes with Pixel-level Details and Materials from Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extracts quad-dominant meshes with Catmull-Clark subdivision, vertex displacement, and PBR materials/lighting from posed multi-view images by end-to-end differentiable re-meshing of a neural SDF.
Problem
Triangle meshes from differentiable image-based 3D reconstruction contain irregular, sliver triangles that are hard to edit and animate, while quad re-meshing is non-differentiable and cannot correct its own approximation errors.
Approach
Jointly learn orientation and position fields with MLPs (self-learning losses with stop-gradient smoothing and a gradient-skipping trick) so that field-aligned quad re-meshing of a Marching Cubes mesh from a neural SDF is differentiable; then apply differentiable Catmull-Clark subdivision and a neural displacement field for pixel-level detail, plus Nvdiffrec-style material and lighting decompositi
Key results
- On NeRF Synthetic, the displaced quad mesh reaches vis. CD 3.0e-2 versus 3.3e-2 for FlexiCubes, with PSNR 27.96 comparable to Nvdiffrec's 29.06.
- On the 79-object Myles dataset, the displaced mesh achieves CD 4.63e-5 (best among methods) and F1 0.70, versus 4.87e-5 for FlexiCubes.
- The quad-dominant mesh has 0.00% aspect ratio outliers above 4 versus 6.69% for FlexiCubes and 16.89% for DMTet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use QUADify's quad-dominant mesh instead of FlexiCubes | QUADify mesh reconstruction · Myles dataset (79 objects) | AR>4 (%): 6.69 -> 0.00 | Helped |
| add one subdivision level and displacement to the quad mesh (ablation) | QUADify · Myles dataset (79 objects) | Chamfer Distance (10−5): 4.96 -> 4.63 | Helped |
| add one subdivision level and displacement to the quad mesh (ablation) | QUADify · NeRF Synthetic | visible Chamfer Distance (10−2): 3.2 -> 3.0 | Helped |
| use a quad-dominant mesh instead of a DMTet triangle mesh before subdivision and displacement | mesh reconstruction · Spot | PSNR: 32.29 | 0.9825 -> 32.79 | 0.9854 | Helped |
| remove orientation and position field self-learning loss Lop (ablation) | QUADify · shape reconstruction and topology alignment | reconstruction quality | Hurt |
| stop skipping gradients through orientation and position field smoothing (ablation) | QUADify · SDF surface reconstruction | reconstruction quality | Hurt |
Surprising findings
- Subdividing and displacing a DMTet triangle mesh creates shading artifacts from irregular skinny triangles, whereas the quad-dominant topology avoids them.
Limitations
- Continuous MLP-based SDF is prone to local minima, especially in concave regions.
- Gold Cape scene: cannot recover a hole in the object, causing a 3 dB PSNR drop.
- Re-meshing coarsens geometry resolution relative to the input triangle mesh and requires a continuous surface during optimization.
Experimental setup
Datasets: NeRF Synthetic NeRD Myles dataset
Benchmarks: NeRF Synthetic NeRD Myles dataset
Metrics: PSNR SSIM LPIPS visible Chamfer Distance Chamfer Distance F1 score aspect ratio outlier percentage radius ratio outlier percentage vertex count
Baselines: Nvdiffrec NvdiffrecMC FlexiCubes DMTet Marching Cubes InstantMeshes
Builds on: Nvdiffrec InstantMeshes Neuralangelo Deep Marching Tetrahedra Field-aligned quad re-meshing (Jakob et al.) Catmull-Clark subdivision (Dupuy et al.)
Closest papers by meaning
- Extracting Triangular 3D Models, Materials, and Lighting From Images (CVPR 2022 · similarity 0.90)
- MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers (ICLR 2025 · similarity 0.87)
- MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers (CVPR 2024 · similarity 0.87)
- ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction (NeurIPS 2025 · similarity 0.86)
- DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representation (NeurIPS 2024 · similarity 0.86)
- BG-Triangle: Bezier Gaussian Triangle for 3D Vectorization and Rendering (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.