Research map / Detection and segmentation / Open-set and few-shot detection
Object-Centric Learning with Slot Mixture Module
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Replaces Slot Attention's k-means-like soft clustering with a learnable Gaussian Mixture Model (Slot Mixture Module) whose slots concatenate cluster means and variances, improving set property prediction on CLEVR and object discovery on ClevrTex.
Problem
Slot Attention acts as a soft k-means and represents each slot only by a cluster center, limiting expressiveness; it also struggles on complex synthetic and real images and is unstable to train.
Approach
A learnable GMM-based slot module (Slot Mixture Module) runs EM-style E and M steps with neural updates, and represents each slot as the concatenation of the cluster mean and diagonal covariance, plus mixture weights to identify empty slots.
Key results
- On CLEVR set property prediction, SMM reaches AP_0.25 98.4 ± 0.7 and AP_0.125 92.1 ± 1.2, beating iDSPN (95.8 and 76.9) at stringent thresholds.
- On ClevrTex object discovery, SMM achieves FG-ARI 74.7 ± 1.3 versus 62.40 ± 2.33 for Slot Attention.
- Replacing SA with SMM in SLATE lowers cross-entropy on all reported datasets (e.g., CLEVR-Mirror 0.82 to 0.20; ShapeStacks 88.3 to 62).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Slot Attention with Slot Mixture Module | SLATE · CLEVR-Mirror test | Cross-Entropy: 0.82 $\pm$ 0.05 -> 0.20 $\pm$ 0.02 | Helped |
| Replace Slot Attention with Slot Mixture Module | SLATE · ShapeStacks test | FID: 56.6 $\pm$ 2.8 -> 50.4 $\pm$ 1.84 | Helped |
| Replace Slot Attention with Slot Mixture Module | SLATE · ClevrTex test | Cross-Entropy: 566 $\pm$ 18.1 -> 507 $\pm$ 21.7 | Helped |
| Replace Slot Attention with Slot Mixture Module | SLATE · COCO test | Cross-Entropy: 540 $\pm$ 17.5 -> 461 $\pm$ 12.1 | Helped |
| Replace Slot Attention with Slot Mixture Module | SLATE · CelebA test | FID: 39.2 $\pm$ 1.5 -> 41.3 $\pm$ 1.6 | Hurt |
| Replace Slot Attention with Slot Mixture Module | SLATE · Bitmoji test | FID: 30.8 $\pm$ 0.52 -> 27.0 $\pm$ 0.57 | Helped |
| Use Slot Mixture Module instead of Slot Attention | Set property prediction model · CLEVR | AP$_{0.25}$: 95.8 $\pm$ 0.7 -> 98.4 $\pm$ 0.7 | Helped |
| Use Slot Mixture Module instead of Slot Attention | Object discovery model · ClevrTex | FG-ARI: 62.40 $\pm$ 2.33 -> 74.7 $\pm$ 1.3 | Helped |
| Use GMM clustering instead of k-means | Vanilla clustering set prediction model · CLEVR set property prediction, 100k iterations | AP$_{0.5}$: 7.2 $\pm$ 1.7 -> 9.2 $\pm$ 1.6 | Helped |
| Use 3 update iterations instead of 7 at test time (ablation) | SMM object discovery model · CLEVR6 test | FG-ARI: 97.1 -> 99.8 | Helped |
| Use 2 update iterations instead of 3 at test time (ablation) | SMM object discovery model · CLEVR6 test | FG-ARI: 99.8 -> 78.9 | Hurt |
| Use 1 update iteration instead of 3 at test time (ablation) | SMM object discovery model · CLEVR6 test | FG-ARI: 99.8 -> 67.0 | Hurt |
Surprising findings
- Slot Attention with detached last-iteration slots and rescaled coordinates still trails iDSPN, indicating the baseline's gap is partly due to training details.
- SMM is less robust than SA to changing the number of update iterations at test time (CLEVR6 FG-ARI drops to 97.1 at 7 iterations vs 99.5 for SA at 7 iterations).
Limitations
- Requires specifying the number of slots in advance, with over/under-segmentation and poor quality for large slot counts.
- Real-world image reconstruction and attention maps remain poor, not aligned with human segmentation on COCO.
- Less robust than Slot Attention to different numbers of update iterations at test time.
Experimental setup
Datasets: CLEVR CLEVR-Mirror ShapeStacks ClevrTex COCO-2017 CelebA Bitmoji CLEVR10 Multi-dSprites Tetrominoes CLEVR6
Benchmarks: CLEVR ClevrTex CLEVR-Mirror ShapeStacks COCO-2017 CelebA Bitmoji CLEVR10 Multi-dSprites Tetrominoes
Metrics: Average Precision (AP) at distance thresholds FID cross-entropy FG-ARI
Baselines: Slot Attention iDSPN SA-MESH K-means clustering SLATE(SA)
Builds on: Slot Attention SLATE iDSPN Spatial Broadcast decoder Expectation Maximization
Closest papers by meaning
- Adaptive Slot Attention: Object Discovery with Dynamic Slot Number (CVPR 2024 · similarity 0.89)
- Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention (NeurIPS 2024 · similarity 0.88)
- Grounded Object-Centric Learning (ICLR 2024 · similarity 0.88)
- MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learning (NeurIPS 2025 · similarity 0.86)
- GLASS: Guided Latent Slot Diffusion for Object-Centric Learning (CVPR 2025 · similarity 0.86)
- Slot State Space Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.