Research map / Point clouds / Shape and surface completion
MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MeshCoder, which trains a point-cloud-conditioned multimodal LLM (Llama-3.2-1B with triplane shape tokenizer) to translate 3D point clouds into editable Blender Python scripts, trained on a ~1M-object synthetic Infinigen-Indoor code dataset.
Problem
Reconstructing complex 3D objects as editable programs is hard because existing shape-program DSLs model only simple primitives and training data with paired object-code is scarce and limited in category and complexity.
Approach
Design expressive Blender Python APIs (translation sweeps, bridge loops, booleans, arrays), use a part-to-code model to label parts of Infinigen-Indoor objects and stitch part codes into object code, then train a triplane-tokenizer-fed LLM to map point clouds directly to Blender scripts.
Key results
- MeshCoder reaches 86.75% average voxel IoU and 0.063e-2 average CD on the Infinigen-Indoor test set, versus 67.62% IoU / 1.87e-2 CD for PLAD and 45.03% IoU / 6.01e-2 CD for Shape2Prog.
- Across 41 categories, MeshCoder achieves lower CD than both baselines in every category shown in the appendix table.
- The paper qualitatively shows that GPT-4o can answer geometry and structure questions from MeshCoder-predicted code, though color information is absent.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MeshCoder instead of PLAD | Shape-to-code reconstruction on Infinigen Indoor · Infinigen Indoor All (Avg.) | L2 CD (× 10^-2): 1.87 -> 0.06 | Helped |
| use MeshCoder instead of PLAD | Shape-to-code reconstruction on Infinigen Indoor · Infinigen Indoor All (Avg.) | Voxel IoU (%): 67.62 -> 86.75 | Helped |
| use MeshCoder instead of Shape2Prog | Shape-to-code reconstruction on Infinigen Indoor · Infinigen Indoor All (Avg.) | L2 CD (× 10^-2): 6.00 -> 0.06 | Helped |
| use MeshCoder instead of Shape2Prog | Shape-to-code reconstruction on Infinigen Indoor · Infinigen Indoor All (Avg.) | Voxel IoU (%): 45.03 -> 86.75 | Helped |
| use MeshCoder instead of PLAD | Shape-to-code reconstruction across object categories · Infinigen Indoor categories | Voxel IoU (%) | Mixed |
Surprising findings
- PLAD outperforms MeshCoder on IoU for a few categories such as Fork (89.28 vs 58.86) and Spoon, and on CD for Fork and Spoon, which the paper does not highlight.
Limitations
- Targets human-made objects; applicability to organic forms such as animals and humans is underdeveloped.
- Inferred code contains only geometry, not color information, so color questions cannot be answered.
- Dataset is synthetic (Infinigen-Indoor) and objects with any part failing reconstruction were discarded.
Experimental setup
Datasets: Infinigen-Indoor Synthetic Part Dataset (procedurally generated by authors)
Benchmarks: Infinigen-Indoor test split
Metrics: L2 Chamfer Distance voxel IoU at 32^3
Baselines: Shape2Prog PLAD
Builds on: Shape2Prog PLAD Infinigen-Indoor LoRA
Closest papers by meaning
- PyTorchGeoNodes: Enabling Differentiable Shape Programs for 3D Shape Reconstruction (CVPR 2025 · similarity 0.86)
- MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers (CVPR 2024 · similarity 0.86)
- PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidance (ICLR 2025 · similarity 0.85)
- MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers (ICLR 2025 · similarity 0.85)
- 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code (NeurIPS 2026 · similarity 0.85)
- BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.