Research map / Detection and segmentation / Open-set and few-shot detection

OBJECT-UNI: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation

NeurIPS 2026 · Detection and segmentation › Open-set and few-shot detection · approach: Object-centric learning theory and video · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Object-Uni is a unified MLLM+DiT model that treats object 3D orientation as a shared variable for pose understanding, pose-controlled generation, and novel view synthesis, trained on the new UniSpatial-80K benchmark with a viewpoint-language abstraction and object-token pose anchor.

Problem

Unified understanding-and-generation models can describe objects in language but cannot precisely represent continuous object poses or generate geometrically consistent images under target viewpoints, and pose estimation and pose-conditioned generation are usually studied separately.

Approach

Treat object pose as an explicit geometric variable shared by understanding and generation; map continuous azimuth/polar/rotation into structured viewpoint words for the MLLM while keeping continuous angle supervision; and ground each pose prediction to an object token (PoseAnchor) for multi-object binding.

Key results

• On KITTI-Cityscapes single-object azimuth, Object-Uni reaches 22.87 error vs 27.33 for Orient Anything V1 (best of baselines) and AUC@30 of 74.44.
• On Objectron pose-controllable generation (single-object), Object-Uni improves mIoU from 40.77 (SceneDesigner) to 78.23.
• On Objectron multi-object pose-controllable generation, mIoU rises from 27.25 to 69.08 versus SceneDesigner.

Surprising findings

• General MLLMs GPT-4o and Gemini-2.5-pro perform poorly on polar and rotation estimation in multi-object scenes on Objectron, sometimes near random.

Limitations

• Weaker at generating fine-grained text and human details, inherited from the generative backbone.
• Pose-controllable generation evaluated only against SceneDesigner; UniSpatial-80K constructed from existing pose-annotated datasets with category-level coverage limits.

Experimental setup

Datasets: KITTI Cityscapes Objectron ImageNet3D OmniNOCS

Benchmarks: UniSpatial-80K (KITTI-Cityscapes, ImageNet3D, Objectron subsets)

Metrics: angular error AUC@5/10/30 mIoU Acc_ls CLIP score

Baselines: GPT-4o Gemini-2.5-pro Orient Anything V1 Orient Anything V2 SceneDesigner

Builds on: Ming-Lite-Uni Orient Anything CNOCS-based 9-DoF pose control Grounding DINO

Closest papers by meaning

• BOP-ASK: Object-Interaction Reasoning for Vision-Language Models (CVPR 2026 · similarity 0.90)
• SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation (NeurIPS 2025 · similarity 0.89)
• SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.89)
• Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation (ICLR 2026 · similarity 0.88)
• CTRL-O: Language-Controllable Object-Centric Visual Representation Learning (CVPR 2025 · similarity 0.87)
• 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.87)