Research map / Detection and segmentation / Open-set and few-shot detection
Interaction Asymmetry: A General Principle for Learning Composable Abstractions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes interaction asymmetry, a principle formalized via block-diagonality of higher-order derivatives of the generator, proves disentanglement and compositional generalization for n=0,1,2, and introduces an attention-regularized Transformer-VAE that matches object-centric baselines on Sprites and CLEVR6.
Problem
Understanding which properties of concepts enable learning representations that are both disentangled and compositionally generalizable, with provable guarantees and a practical method for object-centric learning.
Approach
Formalizes interaction asymmetry: parts of the same concept interact at higher-order (n+1) derivative complexity than parts of different concepts; proves identifiability of disentanglement and compositional generalization for n up to 2, and regularizes cross-attention weights in a Transformer-VAE to minimize cross-slot interactions.
Key results
- On Sprites the regularized Transformer-VAE reaches J-ARI 93.6±0.5 and JIS 95.0±1.7, above the unregularized Transformer (80.5 J-ARI, 57.0 JIS) and Slot Attention with Transformer decoder (90.1 J-ARI, 73.6 JIS).
- On CLEVR6 the regularized model gets J-ARI 96.5±0.3 and JIS 83.8±1.0, versus 92.7 and 54.8 for the unregularized Transformer; Slot Attention with Spatial-Broadcast decoder reaches higher JIS (95.3) and J-ARI (97.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the KL and interaction losses to the Transformer-VAE | Transformer encoder + Transformer decoder · Sprites | J-ARI: 80.5 ± 3.4 -> 93.6 ± 0.5 | Helped |
| Add the KL and interaction losses to the Transformer-VAE | Transformer encoder + Transformer decoder · Sprites | JIS: 57.0 ± 6.5 -> 95.0 ± 1.7 | Helped |
| Add the KL and interaction losses to the Transformer-VAE | Transformer encoder + Transformer decoder · CLEVR6 | J-ARI: 92.7 ± 2.7 -> 96.5 ± 0.3 | Helped |
| Add the KL and interaction losses to the Transformer-VAE | Transformer encoder + Transformer decoder · CLEVR6 | JIS: 54.8 ± 2.9 -> 83.8 ± 1.0 | Helped |
| Add the KL loss (rather than training without it) (ablation) | Transformer-VAE with interaction loss · Sprites | J-ARI: 82.8 ± 2.9 -> 93.6 ± 0.5 | Helped |
| Add the KL loss (rather than training without it) (ablation) | Transformer-VAE with interaction loss · CLEVR6 | J-ARI: 79.2 ± 10.4 -> 96.5 ± 0.3 | Helped |
| Add the KL loss (rather than training without it) (ablation) | Transformer-VAE with interaction loss · CLEVR6 | JIS: 51.6 ± 4.8 -> 83.8 ± 1.0 | Helped |
| Add the interaction loss (rather than training without it) (ablation) | Transformer-VAE with KL loss · Sprites | J-ARI: 92.6 ± 1.6 -> 93.6 ± 0.5 | Helped |
| Add the interaction loss (rather than training without it) (ablation) | Transformer-VAE with KL loss · Sprites | JIS: 92.8 ± 0.7 -> 95.0 ± 1.7 | Helped |
| Add the interaction loss (rather than training without it) (ablation) | Transformer-VAE with KL loss · CLEVR6 | J-ARI: 96.6 ± 0.3 -> 96.5 ± 0.3 | No clear effect |
| Add the interaction loss (rather than training without it) (ablation) | Transformer-VAE with KL loss · CLEVR6 | JIS: 80.3 ± 0.3 -> 83.8 ± 1.0 | Helped |
Surprising findings
- Spatial Broadcast Decoders achieve higher JIS on CLEVR6 but fail to model reflections between objects, while the proposed model captures them.
Limitations
- Compositional generalization is proven theoretically but not empirically tested; method only ensures in-domain properties.
- Theory is proven rigorously only for n up to 2; general n is conjectured.
- Experiments are limited to synthetic object-centric image datasets (Sprites, CLEVR6).
Experimental setup
Datasets: Sprites CLEVR6
Benchmarks: Sprites CLEVR6
Metrics: J-ARI JIS
Baselines: Slot Attention with Spatial-Broadcast decoder Slot Attention with Transformer decoder unregularized Transformer autoencoder
Builds on: Brady et al. 2023 compositional disentanglement Lachapelle et al. additivity and sufficient nonlinearity Slot Attention VAE Transformer
Closest papers by meaning
- Compositional Generalization via Forced Rendering of Disentangled Latents (ICML 2025 · similarity 0.87)
- Object centric architectures enable efficient causal representation learning (ICLR 2024 · similarity 0.87)
- Provable Compositional Generalization for Object-Centric Learning (ICLR 2024 · similarity 0.87)
- Disentangled Representation Learning via Modular Compositional Bias (NeurIPS 2025 · similarity 0.87)
- You Can’t Have It Both Ways: Concept Entanglement Limits Diffusion Model Unlearning (NeurIPS 2026 · similarity 0.86)
- Disentangling Representations through Multi-task Learning (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.