Research map / 3D vision / 3D generation
Orientation Matters: Making 3D Generative Models Orientation-Aligned
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds Objaverse-OA, a 14,832-model orientation-aligned 3D dataset over 1,008 categories, and fine-tunes Trellis and Wonder3D to generate canonically oriented 3D objects from single images.
Problem
Single-image 3D generative models produce objects with inconsistent, uncanonical orientations because their training data (Objaverse, TRELLIS-500K) is orientation-misaligned, which hurts downstream tasks like orientation estimation and rotation manipulation.
Approach
Curate a large category-diverse orientation-aligned dataset (VLM pre-labeling with manual correction) and fine-tune existing 3D generators (Trellis sparse-structure generator only; Wonder3D with LoRA and canonical-view supervision) so they directly output canonically oriented objects, rather than post-hoc alignment.
Key results
- Trellis-OA reduces Chamfer Distance on GSO to 0.0407 (vs 0.0770 for Trellis) and on Toys4k to 0.0393 (vs 0.0951), with best Toys4k LPIPS and CLIP.
- Wonder3D-OA achieves best CD on GSO (0.0564 vs 0.0894 for Wonder3D) and Toys4k (0.0548 vs 0.0932), beating post-hoc PCA, VLM and Orient Anything alignment variants.
- On stick-like ImageNet3D objects, Ours (ViT-L) reaches Acc@30 of 62.25 vs 9.8 for Orient Anything (ViT-L); on Toys4k, Orient Anything (ViT-L) remains best at 63.18 vs 52.87.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Wonder3D on orientation-aligned Objaverse-OA with canonical-view supervision and LoRA | Wonder3D · GSO and Toys4k | CD: 0.0788 -> 0.0564; 0.0858 -> 0.0548 | Helped |
| fine-tune Trellis on orientation-aligned Objaverse-OA | Trellis · GSO and Toys4k | CD: 0.0421 -> 0.0407; 0.0564 -> 0.0393 | Helped |
| fine-tune Trellis on 100 categories with 5720 objects instead of the full Objaverse-OA set (ablation) | Trellis-OA · GSO and Toys4k | CD: 0.0407 -> 0.0448; 0.0393 -> 0.0465 | Hurt |
| fine-tune Trellis on 100 categories with 5720 objects instead of the full Objaverse-OA set (ablation) | Trellis-OA · GSO and Toys4k | LPIPS: 0.2118 -> 0.2224; 0.1932 -> 0.2055 | Hurt |
| fine-tune Trellis on 100 categories with 5720 objects instead of the full Objaverse-OA set (ablation) | Trellis-OA · GSO and Toys4k | CLIP: 88.41 -> 82.46; 95.71 -> 93.74 | Mixed |
| replace LGM with Wonder3D's NeuS lifting module (ablation) | Wonder3D-OA · GSO and Toys4k | CD: 0.0564 -> 0.0609; 0.0548 -> 0.0571 | Hurt |
| fine-tune Wonder3D on Objaverse-OA | Wonder3D · GSO | LPIPS: 0.1998 -> 0.2270 | Mixed |
| use the proposed orientation estimation method with ViT-L instead of Orient Anything (ViT-L) | zero-shot orientation estimation · Stick-like objects from ImageNet3D | Acc@30: 9.8 -> 62.25 | Helped |
| use the proposed orientation estimation method with ViT-L instead of Orient Anything (ViT-L) | zero-shot orientation estimation · Toys4k | Acc@30: 63.18 -> 52.87 | Hurt |
Surprising findings
- On GSO, Trellis + VLM (Gemini-2.0) post-hoc alignment achieves better LPIPS and CLIP than Trellis-OA, which the authors attribute to GSO's irregular geometry and appearance.
- Orient Anything (ViT-L) outperforms the proposed zero-shot orientation estimator on Toys4k in Acc@30 (63.18 vs 52.87).
Limitations
- Manual correction was needed for about 600 categories; VLM front-view recognition falls short of human accuracy on stick-like, thin, and ambiguous objects.
- Failure cases: rear-view inputs yield suboptimal front views, and unseen categories may inherit features from training categories.
- Orientation estimation is not designed for occluded inputs; severe occlusion degrades results.
Experimental setup
Datasets: Objaverse-LVIS Objaverse TRELLIS-500K GSO Toys4k ImageNet3D
Benchmarks: GSO Toys4k ImageNet3D
Metrics: Chamfer Distance LPIPS CLIP score Acc@30 orientation absolute error
Baselines: Wonder3D + PCA Wonder3D + VLM (Gemini-2.0) Wonder3D + Orient Anything Trellis + PCA Trellis + VLM (Gemini-2.0) Trellis + Orient Anything FSDetView Orient Anything (ViT-S) Orient Anything (ViT-L)
Builds on: TRELLIS Wonder3D Orient Anything FoundationPose ImageDream
Closest papers by meaning
- Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models (ICML 2025 · similarity 0.91)
- Orient Anything V2: Unifying Orientation and Rotation Understanding (NeurIPS 2025 · similarity 0.88)
- Evaluating Multiview Object Consistency in Humans and Image Models (NeurIPS 2024 · similarity 0.87)
- One-shot 3D Object Canonicalization based on Geometric and Semantic Consistency (CVPR 2025 · similarity 0.87)
- OBJECT-UNI: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation (NeurIPS 2026 · similarity 0.86)
- 3D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimation (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.