Research map / 3D vision / 3D generation
Flex3D: Feed-Forward 3D Generation with Flexible Reconstruction Model and Input View Curation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Flex3D generates a pool of candidate views with fine-tuned Emu multi-view/video diffusion models, curates them via a back-view quality classifier and LoFTR consistency check, and reconstructs 3D Gaussians with FlexRM, a tri-plane transformer that accepts an arbitrary number of posed views.
Problem
Feed-forward 3D generation from text or a single image relies on multi-view diffusion outputs that are inconsistent or low quality, and reconstructors typically accept a small fixed number of views, limiting quality.
Approach
Generate many candidate views, curate only the high-quality consistent ones via a quality classifier and feature matching, then reconstruct with a flexible transformer (FlexRM) that handles variable view counts, stronger camera conditioning, and tri-plane-to-3DGS decoding with residual position offsets, plus training on simulated imperfect views produced by FlexRM itself.
Key results
- FlexRM with 4 input views reaches PSNR 25.55 on GSO versus 25.03 for GRM, and with 1 view reaches 21.21 vs 19.10 for VFusion3D.
- Flex3D text-to-3D CLIP text similarity 0.277 vs 0.272 for the best baseline (InstantMesh); user study win rate over 92% against all baselines.
- Flex3D's win rates in the user study were 92.5% to 100% across seven baselines, with the reconstructor scaling from PSNR 26.33 at 8 views to 26.77 at 32 views.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase FlexRM input views from 1 to 32 | FlexRM · GSO reconstruction | PSNR: 21.21 -> 26.77 | Helped |
| Add stronger camera conditioning (ablation) | FlexRM · GSO reconstruction, 32 input views | PSNR: +0.3 dB | Helped |
| Add stronger camera conditioning (ablation) | FlexRM · GSO reconstruction, average over 1, 4, 8, and 16 input views | PSNR: 24.31 -> 24.35 | Helped |
| Use positional offsets instead of directly predicting Gaussian positions (ablation) | FlexRM · GSO reconstruction, average over 1, 4, 8, and 16 input views | PSNR: 23.41 -> 24.35 | Helped |
| Add positional offsets (ablation) | FlexRM · GSO reconstruction, average over 1, 4, 8, and 16 input views | PSNR: 22.19 -> 24.35 | Helped |
| Use two-stage training (ablation) | FlexRM · GSO reconstruction, average over 1, 4, 8, and 16 input views | PSNR: 23.38 -> 24.35 | Helped |
| Add multi-view generation at varying elevations (ablation) | Flex3D · 3D generation | CLIP text sim: 0.273 -> 0.277 | Helped |
| Add consistency verification for view selection (ablation) | Flex3D · 3D generation | CLIP text sim: 0.269 -> 0.277 | Helped |
| Add back-view quality assessment (ablation) | Flex3D · 3D generation | CLIP text sim: 0.272 -> 0.277 | Helped |
| Add imperfect input-view simulation (ablation) | Flex3D and FlexRM · 3D generation | CLIP text sim: 0.271 -> 0.277 | Helped |
| Add imperfect input-view simulation (ablation) | Flex3D and FlexRM · GSO reconstruction | PSNR: 24.87 -> 24.90 | Helped |
| Add Gaussian or salt-and-pepper image noise instead of the proposed simulation (ablation) | FlexRM · GSO reconstruction | PSNR | Hurt |
Surprising findings
- Removing stronger camera conditioning has a relatively small effect on average performance, mattering mainly when many views are used.
Limitations
- Extracting clean meshes from 3D Gaussians requires multi-step post-processing.
- Tri-plane representation is limited by resolution and is hard to extend to large scene generation.
- View selection can fail with floaters, small transparent objects, or thin geometries.
Experimental setup
Datasets: Internal Objaverse-like renders (700k objects) Emu-video synthetic dataset (2.7M videos) Google Scanned Objects DreamFusion prompts
Benchmarks: Google Scanned Objects (GSO) DreamFusion prompts (404)
Metrics: PSNR SSIM LPIPS CLIP image similarity Chamfer distance normal correctness CLIP text similarity VideoCLIP text similarity user study win rate
Baselines: OpenLRM VFusion3D LGM InstantMesh GRM LN3Diff 3DTopia-XL LRM
Builds on: Instant3D GS-LRM Emu Emu Video LRM
Closest papers by meaning
- Instant3D: Fast Text-to-3D with Sparse-view Generation and Large Reconstruction Model (ICLR 2024 · similarity 0.90)
- IT3D: Improved Text-to-3D Generation with Explicit View Synthesis (AAAI 2024 · similarity 0.89)
- IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation (ICML 2024 · similarity 0.89)
- HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D (CVPR 2024 · similarity 0.88)
- ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models (CVPR 2024 · similarity 0.88)
- Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.