Research map / 3D vision / 3D generation
UnCommon Objects in 3D
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces uCO3D, a large, diverse, quality-checked dataset of ~170k full-360° object videos from 1,000+ LVIS categories with SfM cameras, point clouds, 3DGS reconstructions and captions, and shows it improves LightplaneLRM, CAT3D-like NVS, and Instant3D-like text-to-3D training.
Problem
Real object-centric 3D datasets either have few objects with accurate scans or many objects with low-quality, partial-view SfM annotations; there is no dataset balancing scale, diversity and quality for 3D learning.
Approach
Crowdsourced full-360° turntable videos across 1k+ LVIS categories, manually verified for coverage, annotated with VGGSfM cameras, densified point clouds, and 3D Gaussian Splat reconstructions that are used to re-render canonical views for training generative models.
Key results
- LightplaneLRM trained on uCO3D reaches 0.093 LPIPS / 24.61 PSNR on OmniObject3D vs 23.62 PSNR for CO3Dv2 and 23.39 for MVImgNet.
- CAT3D-like trained on uCO3D improves PSNR by 3-4 points over MVImgNet on DTU and LLFF-style NVS benchmarks (e.g., DTU 16.97 vs 12.79).
- Instant3D-like text-to-3D trained on uCO3D gets FID 63.9 on real prompts vs 82.8 for a model trained on synthetic renders.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train LightplaneLRM on uCO3D instead of CO3Dv2 | LightplaneLRM · OmniObject3D | LPIPS: 0.095 -> 0.093 | Helped |
| Train LightplaneLRM on uCO3D instead of CO3Dv2 | LightplaneLRM · OmniObject3D | PSNR: 23.62 -> 24.61 | Helped |
| Train LightplaneLRM on uCO3D instead of CO3Dv2 | LightplaneLRM · StanfordORB | LPIPS: 0.056 -> 0.057 | Hurt |
| Train LightplaneLRM on uCO3D instead of CO3Dv2 | LightplaneLRM · StanfordORB | PSNR: 25.617 -> 25.715 | Helped |
| Train CAT3D-like on uCO3D instead of CO3Dv2 | CAT3D-like · RealEstate10K | LPIPS: 0.281 -> 0.278 | Helped |
| Train CAT3D-like on uCO3D instead of CO3Dv2 | CAT3D-like · RealEstate10K | PSNR: 20.02 -> 19.77 | Hurt |
| Train CAT3D-like on uCO3D instead of CO3Dv2 | CAT3D-like · LLFF | LPIPS: 0.418 -> 0.418 | No clear effect |
| Train CAT3D-like on uCO3D instead of CO3Dv2 | CAT3D-like · LLFF | PSNR: 14.95 -> 15.16 | Helped |
| Train CAT3D-like on uCO3D instead of CO3Dv2 | CAT3D-like · DTU | LPIPS: 0.329 -> 0.315 | Helped |
| Train CAT3D-like on uCO3D instead of CO3Dv2 | CAT3D-like · DTU | PSNR: 16.42 -> 16.97 | Helped |
| Train CAT3D-like on uCO3D instead of CO3Dv2 | CAT3D-like · Mip-NeRF 360 | LPIPS: 0.532 -> 0.528 | Helped |
| Train Instant3D-like on uCO3D instead of synthetic data | Instant3D-like · Real prompts | FID: 82.8 -> 63.9 | Helped |
Limitations
- CAT3D was reimplemented from scratch since source code was unavailable, so results depend on the reimplementation.
- uCO3D is smaller in number of scenes than MVImgNet (170k vs 220k).
Experimental setup
Datasets: uCO3D MVImgNet CO3Dv2 Objaverse-like synthetic renders OmniObject3D StanfordORB DTU LLFF RealEstate10K Mip-NeRF 360
Benchmarks: OmniObject3D StanfordORB RealEstate10K LLFF DTU Mip-NeRF 360
Metrics: LPIPS PSNR IoU FID
Baselines: LightplaneLRM trained on MVImgNet LightplaneLRM trained on CO3Dv2 CAT3D-like trained on MVImgNet CAT3D-like trained on CO3Dv2 Instant3D-like trained on synthetic renders
Builds on: CO3D MVImgNet LRM CAT3D Instant3D VGGSfM
Closest papers by meaning
- Uni3D: Exploring Unified 3D Representation at Scale (ICLR 2024 · similarity 0.87)
- Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space (CVPR 2025 · similarity 0.86)
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.85)
- LabelAny3D: Label Any Object 3D in the Wild (NeurIPS 2025 · similarity 0.85)
- Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos (CVPR 2024 · similarity 0.84)
- Towards 3D Objectness Learning in an Open World (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.