Research map / Label-efficient and robust learning / Recognition and long-tail
Bridging Neural and Symbolic Representations with Transitional Dictionary Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Transitional Dictionary Learning, a game-theoretic diffusion model with EM-style online prototype clustering and PPO tuning, that learns part and relation dictionaries unsupervised and outperforms unsupervised part segmentation baselines on abstract compositional datasets.
Problem
Learning symbolic-like compositional representations (visual parts and implicit relations) from abstract visual objects that lack distinct visual features, where existing unsupervised part segmentation methods fail.
Approach
Decompose inputs into visual parts with a game-theoretic diffusion model (players with overlap, resource and norm penalties), and learn reusable 1-ary and 2-ary predicate dictionaries via EM-style online prototype clustering, optionally tuned with PPO using a heuristic shape-score reward.
Key results
- On LineWorld, the method reaches 58.0 CIG and 82.6 shape score versus the best baseline UPD at 36.3 CIG and 42.8 SP.
- On symbol grounding LW-G, it achieves 78.4 IoU and 74.8 relation accuracy, while baselines' relation prediction did not converge.
- In human evaluation, about 65% of baseline decompositions were judged not to be strokes, and SP and CIG correlate positively with human interpretability scores.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add clustering loss to the decomposition-only model (ablation) | TDL · LineWorld and OmniGlot | CIG and SP: 26.3 -> 54.6 CIG, 28.4 -> 68.9 SP; 31.8 -> 61.7 CIG, 23.8 -> 64.8 SP | Helped |
| add higher-order predicate optimization (ablation) | TDL with clustering · LineWorld and OmniGlot | SP: 68.9 -> 71.5; 64.8 -> 67.5 | Helped |
| add PPO tuning with heuristic reward (ablation) | TDL with clustering · LineWorld and OmniGlot | SP: 68.9 -> 82.6; 64.8 -> 77.4 | Helped |
| add attention layers (ablation) | TDL with clustering · LineWorld and OmniGlot | SP: 68.9 -> 70.8; 64.8 -> 66.8 | Helped |
| use latent diffusion (ablation) | TDL with clustering · LineWorld and OmniGlot | SP: 68.9 -> 66.9; 64.8 -> 62.8 | Hurt |
| remove pre-training (ablation) | TDL · ShapeGlot transfer: Bed, Lamp, Sofa, Table | IoU: 67.3 -> 18.1; 61.1 -> 18.3; 62.2 -> 21.5; 68.3 -> 19.9 | Hurt |
Surprising findings
- Baselines' relation prediction heads did not converge on LW-G because their segmentations were incorrect.
- Higher-order predicates and attention gave only marginal ablation gains on the datasets used.
Limitations
- Evaluated only on abstract, simple datasets (LineWorld, OmniGlot, ShapeNet5) where higher-order and attention components show marginal gains.
- Exponential complexity with arity limits use beyond low-arity predicates; sampling used as approximation.
Experimental setup
Datasets: LineWorld OmniGlot ShapeNet5 ShapeGlot
Benchmarks: LineWorld OmniGlot ShapeNet5 LW-G OG-G ShapeGlot transfer
Metrics: IoU clustering information gain shape score MAE top-1 relation accuracy
Baselines: DFF SCOPS UPD AutoEncoder
Builds on: SMLD PPO Unigram Language Model ShapeGlot
Closest papers by meaning
- Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products (NeurIPS 2024 · similarity 0.85)
- Discrete Dictionary-based Decomposition Layer for Structured Representation Learning (NeurIPS 2024 · similarity 0.84)
- In-Context Compositional Learning vis Sparse Coding Transformer (NeurIPS 2025 · similarity 0.83)
- Learning to Compose: Improving Object Centric Learning by Injecting Compositionality (ICLR 2024 · similarity 0.83)
- Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models (ICML 2025 · similarity 0.83)
- Causal Graphical Models for Vision-Language Compositional Understanding (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.