Research map / 3D vision / Face and shape
MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates compact triangle meshes by autoregressively sampling a decoder-only GPT2-medium transformer over a learned residual-quantized vocabulary of triangle embeddings produced by a graph-convolutional encoder.
Problem
Neural-field and voxel generative models produce dense, over-tessellated meshes after iso-surfacing, whereas artist-made meshes are compact with sharp edges; direct triangle mesh generation is underexplored.
Approach
Treat a mesh as a sequence of triangles: learn a graph-convolutional encoder with residual vector quantization over faces (per-vertex code grouping) and a discretized-coordinate 1D ResNet decoder, then train a GPT-style decoder-only transformer to predict the next codebook index autoregressively.
Key results
- On Chair, MeshGPT reaches 43.28 COV and 18.46 FID versus GET3D's 40.85 COV and 81.45 FID, with about 125 vertices and 228 faces versus GET3D's 13725 and 27457.
- Across four categories, MeshGPT improves average coverage by about 9% and FID by about 30 points, per the abstract and introduction.
- In a 49-participant user study (784 responses), users preferred MeshGPT over GET3D for shape quality 68% of the time and for triangulation 73% of the time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use learned geometric tokens instead of naive coordinate tokenization (ablation) | MeshGPT · ShapeNet Chair | COV: 27.50 -> 43.28 | Helped |
| include graph encoder features rather than using codebook indices alone (ablation) | MeshGPT · ShapeNet Chair | COV: 39.24 -> 43.28 | Helped |
| pretrain the transformer on all ShapeNet training shapes (ablation) | MeshGPT · ShapeNet Chair | COV: 36.97 -> 43.28 | Helped |
| compress the mesh token sequence (ablation) | MeshGPT · ShapeNet Chair | COV: 30.98 -> 43.28 | Helped |
| aggregate and quantize embeddings across vertex indices rather than per face (ablation) | MeshGPT · ShapeNet Chair | COV: 23.57 -> 43.28 | Helped |
Surprising findings
- Predicting discretized coordinates as a categorical distribution gave more accurate reconstruction than regressing real values, reducing floating-face artifacts.
- Removing pretraining on all ShapeNet shapes and training per category only lowered performance (overfitting), but the gap was modest on FID (27.54 vs 18.46).
Limitations
- Autoregressive sampling is slow, taking 30 to 90 seconds per mesh.
- Learned tokenization may not scale to scene-scale generation.
- Limited to GPT2-medium due to compute, smaller than modern LLMs.
Experimental setup
Datasets: ShapeNetV2
Benchmarks: ShapeNetV2 Chair ShapeNetV2 Table ShapeNetV2 Bench ShapeNetV2 Lamp
Metrics: COV MMD 1-NNA FID KID average vertices and faces user preference
Baselines: AtlasNet BSPNet PolyGen GET3D
Builds on: PolyGen VQ-VAE residual quantization GPT-2 SAGEConv
Closest papers by meaning
- MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers (ICLR 2025 · similarity 0.89)
- PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidance (ICLR 2025 · similarity 0.89)
- MeshArt: Generating Articulated Meshes with Structure-Guided Transformers (CVPR 2025 · similarity 0.89)
- ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction (NeurIPS 2025 · similarity 0.88)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- QUADify: Extracting Meshes with Pixel-level Details and Materials from Images (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.