Research map / Detection and segmentation / Open-set and few-shot detection
Bootstrapping Top-down Information for Self-modulating Slot Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds a top-down pathway to DINOSAUR-style slot attention that bootstraps discrete semantic codes via vector quantization of slots and modulates a second slot-attention pass with channel and spatial modulation maps.
Problem
Object-centric learning with slot attention is bottom-up and assumes visual features within an object are homogeneous, which fails in complex real-world scenes with diverse object appearances.
Approach
Bootstrap top-down semantic codes from slot outputs using a learned VQ codebook, then self-modulate slot attention's value features with channel-wise (from the code via MLP) and spatial-wise (from centered attention maps) modulation maps, re-running slot attention with shared weights.
Key results
- On MOVI-E, FG-ARI improves from 53.8 (DINOSAUR reproduced) to 59.7, a gain of 5.9.
- On COCO, FG-ARI rises from 34.1 to 37.4 and mBO^i from 31.4 to 33.0 relative to reproduced DINOSAUR.
- Ablation on COCO: full model reaches 37.3 FG-ARI and 32.7 mBO versus 34.8 and 30.5 for DINOSAUR; a DINOSAUR with 6 iterations scores worse (28.5 FG-ARI) than the default 3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the top-down pathway | DINOSAUR · MOVI-C | FG-ARI: 54.7$\pm$4.1 -> 58.9$\pm$5.1 | Helped |
| Add the top-down pathway | DINOSAUR · MOVI-E | FG-ARI: 53.8$\pm$2.1 -> 59.7$\pm$3.1 | Helped |
| Add the top-down pathway | DINOSAUR · COCO | FG-ARI: 34.1$\pm$0.9 -> 37.4$\pm$0.0 | Helped |
| Add the top-down pathway | DINOSAUR · VOC | FG-ARI: 27.0$\pm$3.2 -> 26.7$\pm$4.7 | Hurt |
| Add the top-down pathway | DINOSAUR · VOC | mBOᵢ: 41.2$\pm$3.4 -> 43.9$\pm$2.6 | Helped |
| Use six slot-attention iterations instead of the default three (ablation) | DINOSAUR · COCO | FG-ARI: 28.5 | Hurt |
| Use codebook size 128 instead of 512 (ablation) | Top-down pathway · COCO | FG-ARI: 37.3 -> 33.1 | Hurt |
| Use codebook size 256 instead of 512 (ablation) | Top-down pathway · COCO | FG-ARI: 37.3 -> 32.6 | Hurt |
| Use codebook size 1024 instead of 512 (ablation) | Top-down pathway · COCO | FG-ARI: 37.3 -> 31.4 | Hurt |
| Add vector quantization (ablation) | Top-down pathway · COCO | FG-ARI: 36.3 -> 37.3 | Helped |
| Add channel-wise modulation (ablation) | Top-down pathway · COCO | FG-ARI: 35.1 -> 37.3 | Helped |
| Add attention-map shifting (ablation) | Top-down pathway · COCO | FG-ARI: 35.2 -> 37.3 | Helped |
Surprising findings
- Simply increasing slot attention iterations to six degrades DINOSAUR (28.5 FG-ARI vs default), indicating gains come from self-modulation rather than extra iterations.
- On VOC the method does not improve FG-ARI (26.7 vs 27.0 reproduced), which the authors attribute to single-object images.
Limitations
- Performance depends on codebook size; too small or too large degrades results, requiring perplexity-based selection.
- Codebook size is still a pre-defined hyperparameter.
- FG-ARI does not improve on VOC.
Experimental setup
Datasets: MOVI-C MOVI-E PASCAL VOC 2012 trainaug MS COCO 2017
Benchmarks: MOVI-C MOVI-E PASCAL VOC 2012 MS COCO 2017 CLEVR6
Metrics: FG-ARI mBO mIoU
Baselines: DINOSAUR Slot Attention SLATE SlotDiffusion LSD Rotating Features MaskCut SPOT
Builds on: slot attention DINOSAUR VQ-VAE SPOT
Closest papers by meaning
- Object-Centric Pretraining via Target Encoder Bootstrapping (ICLR 2025 · similarity 0.90)
- Grounded Object-Centric Learning (ICLR 2024 · similarity 0.88)
- Adaptive Slot Attention: Object Discovery with Dynamic Slot Number (CVPR 2024 · similarity 0.88)
- Learning to Compose: Improving Object Centric Learning by Injecting Compositionality (ICLR 2024 · similarity 0.88)
- GLASS: Guided Latent Slot Diffusion for Object-Centric Learning (CVPR 2025 · similarity 0.87)
- MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.