Research map / Detection and segmentation / Open-set and few-shot detection
Latent Noise Segmentation: How Neural Noise Leads to the Emergence of Segmentation and Grouping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that injecting independent noise into the latent space of a VAE/AE trained only on image reconstruction yields unsupervised perceptual grouping and segmentation, evaluated on the new Good Gestalt datasets.
Problem
Unsupervised perceptual grouping and segmentation of images in a way that mirrors human Gestalt grouping, without segmentation labels or task-specific training, and without the fixed slot limits of current object-centric models.
Approach
Independent latent noise perturbs units in proportion to their local derivative in the decoder; differences of noisy reconstructions co-vary for pixels of the same object and not across objects, so clustering these differences segments the image. Trained autoencoders/VAEs are used as-is with no added parameters.
Key results
- The AE and VAE with latent noise reach mean ARI 0.849 across six GG datasets, versus 0.600 for Genesis (best baseline), reported as an improvement of about 24.9%.
- Adding noise in latent space instead of image space improves mean ARI by about 0.506 on average across GG datasets.
- Segmentation performance asymptotes with as few as 5 noise samples (time steps).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add independent noise in latent space rather than reconstruction space (ablation) | AE and VAE · GG datasets (mean) | ARI: +0.506 ARI | Helped |
| add a structured latent prior with β>1 (VAE instead of AE) (ablation) | LNS · GG datasets | ARI | No clear effect |
| add a structured latent prior with β>1 (VAE instead of AE) (ablation) | LNS · Proximity and Illusory Occlusion | ARI: t=4.54, p<0.05; t=5.22, p<0.05 | Mixed |
| reduce segmentation samples from 80 to as few as 5 (ablation) | AE and VAE LNS · GG datasets | ARI: 80 to 5 | No clear effect |
| use a VAE latent prior rather than an AE without a prior (ablation) | LNS · GG datasets, best noise level across tasks | variance of optimal noise level: F=2994.56, p=1.11e^{-8} | Helped |
| move noise from latent space to reconstruction space (ablation) | AE LNS · Gradient Occlusion | ARI: 0.926 -> 0.982 | Helped |
| move noise from latent space to reconstruction space (ablation) | VAE LNS · Continuity | ARI: 0.593 -> 0.611 | Helped |
Surprising findings
- The VAE prefers very small noise levels, while the AE performs well over a wider range of noise, with the AE's optimal noise varying more across datasets.
- Quantitative ARI underrates some outputs: single-pixel shifts from latent perturbation are heavily penalized even when the segmentation is qualitatively correct.
Limitations
- Quantitative ARI penalizes small pixel shifts, so metric can understate qualitative quality.
- Continuity and Closure outputs show noisy artifacts; AE does not find the correct number of objects.
- Theoretical argument relies on local PCA properties of the decoder and is only partly proven; scaling to natural images is only qualitatively shown on CelebA.
Experimental setup
Datasets: Good Gestalt (GG) datasets CelebA
Benchmarks: Good Gestalt (GG) datasets CelebA (qualitative)
Metrics: Adjusted Rand Index (ARI)
Baselines: Genesis Genesis-v2 AE reconstruction clustering VAE reconstruction clustering
Builds on: variational autoencoder GECO Genesis Genesis-v2
Closest papers by meaning
- Learning Hierarchical Image Segmentation For Recognition and By Recognition (ICLR 2024 · similarity 0.86)
- Perceptual Group Tokenizer: Building Perception with Iterative Grouping (ICLR 2024 · similarity 0.85)
- Object segmentation from common fate: Motion energy processing enables human-like zero-shot generalization to random dot stimuli (NeurIPS 2024 · similarity 0.85)
- Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation (NeurIPS 2026 · similarity 0.84)
- Factorized Diffusion Architectures for Unsupervised Image Generation and Segmentation (NeurIPS 2024 · similarity 0.84)
- Vision Transformers Learn Gestalt-Like Figure-Ground Cues from Natural Images (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.