Research map / Image generation / Autoregressive and latent generation
MAR-3D: Progressive Masked Auto-regressor for High-Resolution 3D Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MAR-3D, a pyramid VAE plus cascaded masked auto-regressive diffusion model (MAR-LR and MAR-HR) with condition augmentation that generates high-resolution 3D meshes from a single image in continuous latent space.
Problem
Scaling auto-regressive 3D mesh generation to higher-resolution latent tokens is hard because 3D tokens are unordered, vector quantization loses detail, and transformer generators scale poorly with token count.
Approach
A Pyramid VAE using multi-level point cloud cross-attention produces latents at several token resolutions; a cascaded masked auto-regressor with random-order parallel decoding and diffusion-loss token prediction upscales tokens from 256 to 1024, with Gaussian condition augmentation on low-res tokens to curb compounding error.
Key results
- On GSO, MAR-3D reaches F-Score 0.944, CD 0.351, NC 0.835, versus InstantMesh's 0.923, 0.415, 0.780.
- On OmniObject3D, MAR-3D reports CD 0.364 and F-Score 0.931, outperforming InstantMesh (CD 0.427).
- Ablation: removing condition augmentation raises GSO CD from 0.351 to 0.435, and removing the Pyramid VAE raises it to 0.397.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MAR-3D instead of InstantMesh | single-view 3D generation · GSO | F-Score: 0.923 -> 0.944 | Helped |
| Use MAR-3D instead of InstantMesh | single-view 3D generation · GSO | CD: 0.415 -> 0.351 | Helped |
| Use MAR-3D instead of InstantMesh | single-view 3D generation · GSO | NC: 0.780 -> 0.835 | Helped |
| Use MAR-3D instead of InstantMesh | single-view 3D generation · OmniObject3D | F-Score: 0.918 -> 0.931 | Helped |
| Use MAR-3D instead of InstantMesh | single-view 3D generation · OmniObject3D | CD: 0.427 -> 0.364 | Helped |
| Use MAR-3D instead of InstantMesh | single-view 3D generation · OmniObject3D | NC: 0.779 -> 0.826 | Helped |
| Add Pyramid VAE (ablation) | MAR-3D · GSO | F-Score: 0.928 -> 0.944 | Helped |
| Add condition augmentation (ablation) | MAR-3D · GSO | F-Score: 0.902 -> 0.944 | Helped |
| Add MAR-HR (ablation) | MAR-3D · GSO | F-Score: 0.921 -> 0.944 | Helped |
| Add rotation augmentation (ablation) | MAR-3D · GSO | F-Score: 0.934 -> 0.944 | Helped |
Surprising findings
- Directly training MAR or DiT on 1024 tokens degrades performance due to convergence issues under limited GPUs, motivating the cascaded design.
Limitations
- Evaluation limited to 200 shapes from two datasets with single-view input.
- Compute and training details deferred to supplementary material.
Experimental setup
Datasets: Objaverse
Benchmarks: GSO OmniObject3D
Metrics: F-Score Chamfer distance Normal Consistency
Baselines: InstantMesh LGM TripoSR CraftsMan DiT
Builds on: MAR MaskGIT 3DShape2VecSet VAR Objaverse
Closest papers by meaning
- EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generation (ICLR 2025 · similarity 0.88)
- LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion (ICLR 2025 · similarity 0.87)
- Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attention (NeurIPS 2025 · similarity 0.87)
- Masked Autoencoders Are Effective Tokenizers for Diffusion Models (ICML 2025 · similarity 0.87)
- ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction (NeurIPS 2025 · similarity 0.87)
- Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.