Research map / Detection and segmentation / Open-set and few-shot detection
Learning to Compose: Improving Object Centric Learning by Injecting Compositionality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds a composition path to auto-encoding object-centric learning: mixes slots from two images, decodes the composite, and maximizes its likelihood under the diffusion decoder used as a generative prior, improving slot attention segmentation and robustness.
Problem
Auto-encoding objectives used in slot-attention object-centric learning do not directly encourage object-level compositional representations, making results sensitive to slot count, encoder and decoder architecture.
Approach
Extract slots from two images with shared initialization, compose a mixed slot set by exclusive partition, decode it with a one-shot surrogate decoder, and optimize the encoder to maximize the composite image's likelihood under the diffusion decoder used as a generative prior (score-distillation style), alongside an attention-based regularizer.
Key results
- On CLEVRTex, the full method reaches 88.15 FG-ARI, 75.30 mIoU and 75.64 mBO, versus 42.48 / 52.26 / 52.41 for the auto-encoding-only variant (ablation table).
- FG-ARI improves 8 to 29% over baselines on all datasets except PTR, and mIoU/mBO improve on all datasets.
- The method stays robust across 11-17 slots, a weaker CNN encoder, and 133-200% decoder capacity, where baselines degrade substantially.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the composition path with the generative prior (ablation) | Auto-encoding-only LSD · CLEVRTex | FG-ARI: 42.48 -> 65.76 | Helped |
| Add the composition path with the generative prior (ablation) | Auto-encoding-only LSD · CLEVRTex | mIoU: 52.26 -> 67.72 | Helped |
| Add the composition path with the generative prior (ablation) | Auto-encoding-only LSD · CLEVRTex | mBO: 52.41 -> 67.62 | Helped |
| Share slot initialization between images (ablation) | Composition path with prior and no regularization · CLEVRTex | mIoU: 67.72 -> 69.08 | Helped |
| Share slot initialization between images (ablation) | Composition path with prior and no regularization · CLEVRTex | mBO: 67.62 -> 69.28 | Helped |
| Add slot-mask regularization alongside the generative prior (ablation) | Composition path with shared slot initialization · CLEVRTex | FG-ARI: 70.29 -> 88.15 | Helped |
| Add slot-mask regularization alongside the generative prior (ablation) | Composition path with shared slot initialization · CLEVRTex | mIoU: 69.08 -> 75.30 | Helped |
| Add slot-mask regularization alongside the generative prior (ablation) | Composition path with shared slot initialization · CLEVRTex | mBO: 69.28 -> 75.64 | Helped |
| Add slot-mask regularization without the generative prior (ablation) | Composition path with shared slot initialization · CLEVRTex | mIoU: 69.08 -> 58.81 | Hurt |
Surprising findings
- Adding the regularization loss alone on the composition path does not improve performance; it helps only when combined with the generative prior loss.
- A one-step Tweedie denoising estimate of the diffusion decoder gave severe degradation, so a separate weak one-shot surrogate decoder was used instead.
Limitations
- Relies on a surrogate one-shot decoder whose generation quality is much weaker than the diffusion decoder.
- Real-world experiments require bootstrapping with pretrained DINOv2 and Stable Diffusion and multi-stage training.
- PTR FG-ARI does not improve over baselines.
Experimental setup
Datasets: CLEVRTex MultiShapeNet PTR Super-CLEVR BDD100k
Benchmarks: CLEVRTex MultiShapeNet PTR Super-CLEVR BDD100k
Metrics: FG-ARI mIoU mBO object property prediction accuracy/MSE
Baselines: SLATE+ LSD Stable-LSD
Builds on: Slot Attention LSD (latent slot diffusion) DreamFusion-style score distillation SLATE
Closest papers by meaning
- Provable Compositional Generalization for Object-Centric Learning (ICLR 2024 · similarity 0.92)
- Unlocking Compositional Generalization in Continual Few-Shot Learning (NeurIPS 2026 · similarity 0.90)
- Not Just Object, But State: Compositional Incremental Learning without Forgetting (NeurIPS 2024 · similarity 0.88)
- Does Data Scaling Lead to Visual Compositional Generalization? (ICML 2025 · similarity 0.88)
- Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models (CVPR 2025 · similarity 0.88)
- Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.