Research map / 3D vision / Face and shape
Locally Adaptive Neural 3D Morphable Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAMM, a transformer/MLP-Mixer autoencoder for 3D meshes that takes sparse control-vertex displacements as extra decoder input, enabling direct local geometry editing in a single forward pass.
Problem
Learning disentangled 3D morphable models that allow fine, direct local control of dense mesh geometry, while scaling to high-resolution meshes with low memory and fast inference, which GCN-based autoencoders struggle with.
Approach
An autoencoder with region-specific tokenization and a decoder that adds learned embeddings of sparse control-vertex displacements to decoder tokens, trained self-supervised by pairing source and target meshes and using a multilayer loss; disentanglement emerges without explicit disentanglement losses or partitioning the latent code.
Key results
- LAMM-MLPMixer reaches 7.97 mean error on UHM12k reconstruction vs 11.53 for SpiralNet++ (latent size 256).
- With 72k-vertex meshes and batch 32, LAMM uses 7.5GB GPU memory and runs at 0.045s on one CPU thread vs SpiralNet++ at >40GB and 0.58s.
- Generates 12k-vertex meshes at >60fps on a single CPU thread.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LAMM-MLPMixer instead of PCA | 3D shape reconstruction · UHM12k | mean Euclidean distance (×10−2mm): 10.42 -> 7.97 | Helped |
| Use LAMM-Transformer instead of PCA | 3D shape reconstruction · UHM12k+expr. | mean Euclidean distance (×10−2mm): 11.49 -> 9.09 | Helped |
| Use LAMM-MLPMixer instead of PCA | 3D shape reconstruction · UHM | mean Euclidean distance (×10−2mm): 11.73 -> 11.48 | Helped |
| Use LAMM-MLPMixer instead of PCA | 3D shape reconstruction · Handy | mean Euclidean distance (×10−2mm): 25.90 -> 24.60 | Helped |
| Use MLPMixer rather than Transformer backbone (ablation) | LAMM · UHM12k | AE mean Euclidean distance (×10−2mm): 8.79 -> 8.36 | Helped |
| Use MLPMixer rather than Transformer backbone (ablation) | LAMM · UHM12k+expr. | AE mean Euclidean distance (×10−2mm): 8.61 -> 8.98 | Hurt |
| Use MLPMixer rather than Transformer backbone (ablation) | LAMM · UHM | AE mean Euclidean distance (×10−2mm): 15.86 -> 11.63 | Helped |
| Use MLPMixer rather than Transformer backbone (ablation) | LAMM · Handy | AE mean Euclidean distance (×10−2mm): 26.78 -> 24.59 | Helped |
| Train for expression manipulation rather than autoencoding only | LAMM-Transformer · UHM12k+expr. | AE mean Euclidean distance (×10−2mm): 9.08 -> 7.79 | Helped |
Surprising findings
- Autoencoder performance can improve when the model is trained for manipulation, and expression training can increase AE performance significantly.
Limitations
- Hand reconstruction and manipulation errors are larger and concentrated near the wrist where no control points exist.
- Region swapping leaves boundary discontinuities due to restricted rigid alignment.
Experimental setup
Datasets: UHM12k (6k heads registered from UHM data, expanded with 28 blendshapes) UHM 72k-vertex meshes (10k sampled identities) Handy hand model samples (10k hands)
Benchmarks: UHM12k UHM12k+expr. UHM Handy
Metrics: mean Euclidean distance (x10^-2 mm) inference time GPU memory
Baselines: PCA COMA Neural3DMM SpiralNet++ SD-VAE LED-VAE
Builds on: Vision Transformer MLP-Mixer 3D Morphable Model SpiralNet++
Closest papers by meaning
- Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space (CVPR 2025 · similarity 0.86)
- MeshFormer : High-Quality Mesh Generation with 3D-Guided Reconstruction Model (NeurIPS 2024 · similarity 0.86)
- Flatten Anything: Unsupervised Neural Surface Parameterization (NeurIPS 2024 · similarity 0.86)
- GALA: Geometry-Aware Local Adaptive Grids for Detailed 3D Generation (ICLR 2025 · similarity 0.85)
- LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion (ICLR 2025 · similarity 0.85)
- Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modeling (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.