Research map / 3D vision / 3D generation
Identifiable Object Representations under Spatial Ambiguities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VISA, a multi-view probabilistic slot attention method that learns identifiable, view-invariant object content and disentangled viewpoint representations without viewpoint annotations, with identifiability theory and experiments on CLEVR-mv, GQN, GSO and new mv-MoViC/mv-MoViD datasets.
Problem
Obtaining modular object-centric representations under spatial ambiguities such as occlusion and view changes, where single-view object-centric learning fails and multi-view methods often need viewpoint annotations or lack identifiability guarantees.
Approach
Aggregate view-specific slot GMMs obtained after inverse view transformation into a view-invariant content GMM via a convex combination over views, with a learned global view GMM prior and a reconstruction-based ELBO; identifiability is proven up to affine/permutation equivalence without view supervision.
Key results
- On CLEVR-mv, VISA reaches SMCC 0.67 ± .01 and MCC 0.60 ± .04, versus 0.63 SMCC for OCLOC and 0.61 SMCC for MulMON.
- On mv-MoViC in-domain with a transformer decoder, VISA-Transformer obtains mBO 0.38 versus 0.37 for PSA-Transformer and 0.34 for SA-Transformer.
- On the synthetic 2D dataset, the aggregate content distribution across runs has SMCC 0.72 ± 0.04, which the authors read as evidence of identifiability up to affine transformation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VISA instead of OCLOC | object-centric representations · CLEVR-mv | SMCC: $0.63 \pm .02$ -> $0.67 \pm .01$ | Helped |
| Use VISA instead of OCLOC | object-centric representations · CLEVR-mv | MCC: $0.48 \pm .04$ -> $0.60 \pm .04$ | Helped |
| Use VISA instead of OCLOC | object-centric representations · GQN | SMCC: $0.60 \pm .03$ -> $0.59 \pm .01$ | No clear effect |
| Use VISA instead of OCLOC | object-centric representations · GSO | inv-SMCC: $0.54 \pm 0.03$ -> $0.61 \pm .02$ | Helped |
| Use VISA-MLP instead of PSA-MLP | mv-MoViC object-centric model · mv-MoViC in-domain | SMCC: $0.38 \pm 0.002$ -> $0.52 \pm 0.021$ | Helped |
| Use VISA-MLP instead of PSA-MLP | mv-MoViC object-centric model · mv-MoViC out-of-domain | SMCC: $0.40 \pm 0.005$ -> $0.51 \pm 0.029$ | Helped |
| Use VISA-Transformer instead of PSA-Transformer | mv-MoViC object-centric model · mv-MoViC in-domain | SMCC: $0.38 \pm 0.007$ -> $0.44 \pm 0.003$ | Helped |
| Use VISA-Transformer instead of PSA-Transformer | mv-MoViC object-centric model · mv-MoViC out-of-domain | SMCC: $0.39 \pm 0.016$ -> $0.46 \pm 0.033$ | Helped |
| Replace the MLP decoder with a transformer decoder (ablation) | VISA · mv-MoViC in-domain | SMCC: $0.52 \pm 0.021$ -> $0.44 \pm 0.003$ | Hurt |
| Replace the MLP decoder with a transformer decoder (ablation) | VISA · mv-MoViC out-of-domain | SMCC: $0.51 \pm 0.029$ -> $0.46 \pm 0.033$ | Hurt |
Surprising findings
- On mv-MoViC in-domain with the MLP decoder, VISA-MLP has lower mBO (0.28) than PSA-MLP (0.30), so the identifiability gains do not translate into better object segmentation there.
Limitations
- Viewpoint sufficiency and weak injectivity assumptions are strong and may not hold in practice.
- Piecewise-affine mixing functions may be misspecified for real-world data.
- Multi-view identifiability results have been validated mainly on simple synthetic and benchmark datasets.
Experimental setup
Datasets: CLEVR-mv CLEVR-aug GQN GSO mv-MoViC mv-MoViD synthetic 2D GMM illustrative dataset
Benchmarks: CLEVR-mv GQN GSO mv-MoViC
Metrics: SMCC inv-SMCC MCC mBO
Baselines: AE SA PSA MulMON OCLOC
Builds on: Probabilistic Slot Attention Slot Attention MulMON OCLOC DINOSAUR
Closest papers by meaning
- Multi-View Causal Representation Learning with Partial Observability (ICLR 2024 · similarity 0.85)
- Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention (NeurIPS 2024 · similarity 0.85)
- Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs (AAAI 2026 · similarity 0.85)
- Multi-view Relational Distillation for Spatial Reasoning with Vision-Language Models (NeurIPS 2026 · similarity 0.85)
- SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.85)
- Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.