Research map / Image generation / Autoregressive and latent generation
GLASS: Guided Latent Slot Diffusion for Object-Centric Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GLASS, a slot-attention model trained on Stable Diffusion-generated images with semantic (diffusion cross-/self-attention masks) and instance (feature reconstruction) guidance, achieving SOTA object discovery and conditional/compositional generation on real-world scenes.
Problem
Object-centric learning with slot attention performs poorly on real-world multi-object scenes, suffering from over-segmentation, under-segmentation and imprecise boundaries (part-whole ambiguity), and existing slot models cannot generate realistic scenes compositionally.
Approach
Train slot attention on images generated by a pre-trained diffusion model, using the diffusion decoder's cross- and self-attention maps as pseudo ground-truth semantic masks (semantic guidance) combined with MLP reconstruction of encoder features (instance guidance), matched via Hungarian matching; the diffusion decoder also reconstructs/generates images from slots.
Key results
- On VOC instance-level object discovery, GLASS reaches 58.1% mIoUi vs 48.8% for SPOT (+9.3 points).
- Conditional generation FID on COCO improves from 140.62 (StableLSD) to 71.30 for GLASS.
- Zero-shot object discovery on CLEVRTex reaches 46.1 mIoUi vs 39.5 for SPOT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add semantic guidance loss to reconstruction training (ablation) | GLASS · COCO | mIoUi: 30.0 -> 30.9 | Helped |
| Add semantic guidance loss to reconstruction training (ablation) | GLASS · VOC | mIoUi: 34.4 -> 55.1 | Helped |
| Add instance guidance loss to reconstruction training (ablation) | GLASS · COCO | mIoUi: 30.0 -> 29.3 | Hurt |
| Add instance guidance loss to reconstruction training (ablation) | GLASS · VOC | mIoUi: 34.4 -> 38.9 | Helped |
| Add instance guidance alongside semantic guidance (ablation) | GLASS · COCO | mIoUi: 30.9 -> 38.9 | Helped |
| Add instance guidance alongside semantic guidance (ablation) | GLASS · VOC | mIoUi: 55.1 -> 58.1 | Helped |
| Replace DINOv2 encoder with MAE (ablation) | GLASS · COCO | mIoUi: 38.9 -> 30.0 | Hurt |
| Replace DINOv2 encoder with MAE (ablation) | GLASS · VOC | mIoUi: 58.1 -> 41.1 | Hurt |
| Replace pseudo-ground-truth semantic masks with SAMv2 masks (ablation) | GLASS with semantic guidance only · COCO | mIoUi: 32.2 -> 30.4 | Hurt |
| Replace pseudo-ground-truth semantic masks with SAMv2 masks (ablation) | GLASS with semantic guidance only · VOC | mIoUi: 55.6 -> 42.1 | Hurt |
| Use ground-truth class labels instead of generated captions | GLASS · COCO object discovery | mIoUi: 38.9 -> 39.0 | No clear effect |
| Use ground-truth class labels instead of generated captions | GLASS · VOC object discovery | mIoUi: 58.1 -> 57.8 | Hurt |
Surprising findings
- Using generated captions (BLIP-2) as weak supervision outperforms weakly-supervised variants of StableLSD that use bounding boxes or object counts.
Limitations
- Sensitive to the number of slots
- Sometimes binds slots to semantics rather than instances
- Image generation quality still below state-of-the-art generative models
Experimental setup
Datasets: COCO Stable Diffusion generated images
Benchmarks: COCO VOC CLEVRTex Object365
Metrics: mIoUi mBOi mBOc SO-PO-GO PSNR SSIM LPIPS FID accuracy MSE
Baselines: DINOSAUR StableLSD SPOT SlotDiffusion SLATE Slot Attention Dataset Diffusion DiffCut
Builds on: slot attention DINOSAUR StableLSD Dataset Diffusion Stable Diffusion
Closest papers by meaning
- Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generation (ICLR 2025 · similarity 0.93)
- Grounded Object-Centric Learning (ICLR 2024 · similarity 0.91)
- Adaptive Slot Attention: Object Discovery with Dynamic Slot Number (CVPR 2024 · similarity 0.90)
- MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learning (NeurIPS 2025 · similarity 0.89)
- Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention (NeurIPS 2024 · similarity 0.89)
- CTRL-O: Language-Controllable Object-Centric Visual Representation Learning (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.