Research map / Image generation / Autoregressive and latent generation
LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaGeM, a U-Net-style hierarchical VecSet autoencoder with standardization bottleneck and a cascaded latent diffusion model over its multi-level latents, trained faster and with less memory than VecSet on ShapeNet and Objaverse.
Problem
VecSet-based 3D shape autoencoders are limited in expressiveness by the number of latent vectors, overfit on small datasets like ShapeNet, and are costly to scale to large datasets such as Objaverse because self-attention cost is quadratic in the number of vectors.
Approach
A multi-level (U-Net/NVAE-inspired) VecSet autoencoder where cross-attention acts as down/up-sampling between levels, uses per-vector standardization (zero mean, unit variance) instead of KL regularization in the bottleneck, and a cascaded latent diffusion model generates the levels coarse-to-fine, each conditioned on previous levels.
Key results
- On Objaverse-trained LaGeM evaluated on ShapeNet, mean Chamfer (all categories) drops from 4.68 (VecSet) to 2.42 (x100), about 50% improvement.
- Training at 2k latent vectors uses 0.7x time and 0.58x memory relative to VecSet (per paper's Table of running statistics).
- LaGeM-ShapeNet matches VecSet parameter count with slightly better averaged Chamfer and lower training cost on ShapeNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the hierarchical VecSet representation on ShapeNet rather than using single-level VecSet | LaGeM-ShapeNet · ShapeNet mean (all) | Chamfer (× 100): 4.68 -> 4.63 | Helped |
| Train the hierarchical VecSet representation on Objaverse | LaGeM · ShapeNet mean (all) | Chamfer (× 100): 4.68 -> 2.42 | Helped |
| Train the hierarchical VecSet representation on Objaverse | LaGeM · ShapeNet mean (all) | F-Score (× 100): 93.25 -> 98.93 | Helped |
| Train the hierarchical VecSet representation on Objaverse | LaGeM-Objaverse · Thingi10k, ABO, ShapeNet-test, EGAD, GSO, pix3d, FAUST | Chamfer (× 100): 4.52 -> 2.99; 4.91 -> 3.66; 3.25 -> 2.33; 3.27 -> 2.82; 3.78 -> 2.35; 6.53 -> 6.02; 2.10 -> 1.31 | Helped |
| Use hierarchical latent VecSets instead of VecSet | LaGeM · 2k latent vectors, batch size 8 | Training Memory (M): OOM -> 53,791 | Helped |
| Use hierarchical latent VecSets instead of VecSet | LaGeM · 2k latent vectors, batch size 4 | Training Memory (M): 54,543 -> 31,662 | Helped |
| Use hierarchical latent VecSets instead of VecSet | LaGeM · 2k latent vectors, batch size 4 | Training Iteration (sec): 0.6902 -> 0.4853 | Helped |
Surprising findings
- Objaverse-trained model generalizes to ShapeNet with larger gains than the ShapeNet-trained model, suggesting VecSet overfits small datasets.
Limitations
- Training a diffusion model across all latent levels remains expensive; the method does not solve diffusion's high training cost.
- Generative results are preliminary, with limited qualitative evaluation on Objaverse-10k and ShapeNet.
Experimental setup
Datasets: Objaverse ShapeNet Thingi10k ABO EGAD GSO pix3d FAUST
Benchmarks: ShapeNet Thingi10k ABO EGAD GSO pix3d FAUST
Metrics: Chamfer distance F-score training time training memory
Baselines: 3DShape2VecSet
Builds on: 3DShape2VecSet NVAE Cascaded Diffusion EDM DiT
Closest papers by meaning
- LT3SD: Latent Trees for 3D Scene Diffusion (CVPR 2025 · similarity 0.89)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space (ICLR 2024 · similarity 0.88)
- Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representation (ICLR 2025 · similarity 0.87)
- Large-Vocabulary 3D Diffusion Model with Transformer (ICLR 2024 · similarity 0.87)
- MAR-3D: Progressive Masked Auto-regressor for High-Resolution 3D Generation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.