Research map / Image generation / Text-to-image diffusion
ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ORIGEN, a zero-shot test-time method that steers a one-step FLUX-Schnell text-to-image flow model toward 3D object orientations by Langevin-dynamics sampling in latent space guided by an OrientAnything reward, with reward-adaptive time rescaling.
Problem
Text-to-image generation lacks control over 3D orientation of multiple objects; existing orientation-conditioned models are trained on synthetic single-object renders and are limited in azimuth range and object count.
Approach
Reformulate reward-guided latent search as sampling rather than optimization: simulate reward-guided Langevin dynamics on the initial noise of a one-step generator (gradient ascent plus injected Gaussian noise), and add a reward-dependent monitor function that rescales time to speed convergence.
Key results
- On ORIBENCH-Single, ORIGEN reaches 0.871 Acc.@22.5° and 17.41 azimuth error, versus 0.499/59.03 for Zero-1-to-3 and 0.796/20.56 for ReNO.
- On ORIBENCH-Multi, ORIGEN achieves 0.692 Acc.@22.5° versus 0.591 for the best multi-step guidance baseline (FreeDoM).
- In a user study with 100 MTurk participants, ORIGEN was preferred 58.18% of the time versus 20.58% (Zero-1-to-3) and 21.24% (C3DW).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ORIGEN reward-guided sampling instead of ReNO latent optimization | FLUX-Schnell on ORIBENCH-Single · ORIBENCH-Single | Acc.@22.5°: 0.796 -> 0.871 | Helped |
| use ORIGEN reward-guided sampling instead of the strongest baseline | ORIBEN on ORIBENCH-Multi · ORIBENCH-Multi | Acc.@22.5°: 0.591 -> 0.692 | Helped |
| add reward-adaptive time rescaling (ablation) | ORIGEN on ORIBENCH-Single · ORIBENCH-Single | Acc.@22.5°: 0.854 -> 0.871 | Helped |
| add reward-adaptive time rescaling (ablation) | ORIGEN on ORIBENCH-Multi · ORIBENCH-Multi | Acc.@22.5°: 0.679 -> 0.692 | Helped |
| add reward-adaptive time rescaling (ablation) | ORIGEN on ORIBENCH-Single · ORIBENCH-Single | Mean NFE to success criterion: 14.2 -> 12.8 | Helped |
| add reward-adaptive time rescaling (ablation) | ORIGEN on ORIBENCH-Single · ORIBENCH-Single | CLIP: 0.265 -> 0.265 | No clear effect |
| add reward-adaptive time rescaling (ablation) | ORIGEN on ORIBENCH-Single · ORIBENCH-Single | PickScore: 0.224 -> 0.224 | No clear effect |
| use ORIGEN instead of Zero-1-to-3 or C3DW | orientation-grounded image generation in the user study · User study | User Preferences (%): 20.58, 21.24 -> 58.18 | Helped |
Surprising findings
- Vanilla text-to-image models with orientation phrases in the prompt (e.g., FLUX-Schnell) reach only 31.2% 3-view alignment, far below ORIGEN's 82.4%.
- Multi-step training-free guidance methods (DPS, MPGD, FreeDoM) perform poorly on orientation, attributed to blurry early-stage predictions.
Limitations
- Performance is bounded by the quality of the pretrained orientation discriminator (OrientAnything).
- Back-facing generation required adding explicit 'facing back' prompt phrases, since the base model struggled with it.
- Comparisons with prior orientation-conditioned models are restricted to single-object, front-180° azimuth cases.
Experimental setup
Datasets: MS-COCO validation set
Benchmarks: ORIBENCH-Single ORIBENCH-Multi four primitive views benchmark general orientation benchmark
Metrics: Acc.@22.5° absolute azimuth error CLIP Score VQA-Score PickScore user preference
Baselines: Zero-1-to-3 Continuous 3D Words DPS MPGD FreeDoM ReNO SD-Turbo SDXL-Turbo FLUX-Schnell
Builds on: ReNO Leroy et al. time-rescaled SDE OrientAnything GroundingDINO FLUX
Closest papers by meaning
- R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation (ICLR 2024 · similarity 0.87)
- Walking the Schrödinger Bridge: A Direct Trajectory for Text-to-3D Generation (NeurIPS 2025 · similarity 0.86)
- SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding (CVPR 2025 · similarity 0.86)
- GrounDiT: Grounding Diffusion Transformers via Noisy Patch Transplantation (NeurIPS 2024 · similarity 0.85)
- Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior (CVPR 2024 · similarity 0.85)
- Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.