Research map / Label-efficient and robust learning / Recognition and long-tail
Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Soft TPR, a continuous relaxation of Smolensky's Tensor Product Representation, and a Soft TPR Autoencoder that learns distributed compositional visual representations with weak (match-pairing) supervision.
Problem
Symbolic, concatenative compositional representations used in disentanglement clash with the continuous, distributed nature of deep learning, while classic TPRs impose a rigid discrete form that is hard to learn and limited to formal domains like language.
Approach
Relax the TPR to any vector within an epsilon-neighbourhood of an explicit TPR, learned by an autoencoder whose encoder output is penalised toward its greedily quantised explicit TPR (unbinding, VQ-VAE quantisation, TPR construction) plus weakly supervised match-pairing swap losses.
Key results
- Soft TPR reaches DCI 0.863 on Cars3D and 0.828 on MPI3D, a 74% DCI improvement on MPI3D over baselines, with the best FactorVAE scores on all three datasets.
- Abstract visual reasoning accuracy at 100 iterations is 0.804 versus 0.631 for the best baseline (Ada-GVAE-k).
- Soft TPR gives 0.490 R^2 on MPI3D with 100 samples versus 0.206 for the best baseline, a 138% raw improvement in the low-sample regime.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Soft TPR Autoencoder instead of symbolic disentanglement baselines | visual representation learning · Cars3D, Shapes3D, MPI3D | DCI score: Cars3D: 0.664 $\pm$ 0.167 -> 0.863 $\pm$ 0.027; Shapes3D: 0.963 $\pm$ 0.077 -> 0.926 $\pm$ 0.028; MPI3D: 0.475 $\pm$ 0.0 | Mixed |
| train with Soft TPR Autoencoder instead of the strongest reported baseline | representation learner · Cars3D, Shapes3D, MPI3D (100 iterations) | FoV regression $R^{2}$: Cars3D: 0.484 $\pm$ 0.477 -> 0.531 $\pm$ 0.054; Shapes3D: 0.886 $\pm$ 0.033 -> 0.981 $\pm$ 0.003; MPI3D: 0.557 $\pm$ 0.0 | Helped |
| use Soft TPR Autoencoder instead of the strongest reported baseline | downstream abstract visual reasoning model · Abstract visual reasoning (100 iterations) | accuracy: 0.631 $\pm$ 0.037 -> 0.804 $\pm$ 0.016 | Helped |
| use Soft TPR representations instead of TPR representations | downstream FoV regression model · Cars3D (100 samples), Shapes3D (250 samples), MPI3D (100 samples) | FoV regression $R^{2}$: Cars3D: 0.361 $\pm$ 0.031 -> 0.638 $\pm$ 0.010; Shapes3D: 0.669 $\pm$ 0.108 -> 0.728 $\pm$ 0.021; MPI3D: 0.415 $\pm$ 0.0 | Helped |
| remove relaxed representational constraints (ablation) | Soft TPR Autoencoder · MPI3D | DCI score: 0.828 $\pm$ 0.015 -> 0.598 $\pm$ 0.093 | Hurt |
| remove distributed encoding (ablation) | Soft TPR Autoencoder · MPI3D | DCI score: 0.828 $\pm$ 0.015 -> 0.704 $\pm$ 0.135 | Hurt |
| remove both relaxed representational constraints and distributed encoding (ablation) | Soft TPR Autoencoder · MPI3D | DCI score: 0.828 $\pm$ 0.015 -> 0.225 $\pm$ 0.034 | Hurt |
| increase baseline parameter counts | relevant symbolic baselines · disentanglement benchmarks | disentanglement performance | No clear effect |
Surprising findings
- Stricter enforcement of the explicit TPR form (large form-penalty weight) substantially impairs disentanglement, suggesting relaxed constraints help learning.
- Replacing Soft TPRs with their quantised explicit TPRs in downstream tasks removes the downstream advantage.
Limitations
- Assumes role embeddings are linearly independent and known (roles from a fixed, flat set of FoV types).
- Evaluated only on synthetic, weakly supervised visual disentanglement datasets with fixed numbers of FoVs.
- Hierarchical compositional structure is not handled.
Experimental setup
Datasets: Cars3D MPI3D Shapes3D
Benchmarks: Cars3D MPI3D Shapes3D
Metrics: FactorVAE score DCI score R^2 sample efficiency accuracy
Baselines: Ada-GVAE-k GVAE ML-VAE SlowVAE Shu COMET VCT
Builds on: Tensor Product Representation VQ-VAE Ada-GVAE ML-VAE
Closest papers by meaning
- Compositional Generalization via Forced Rendering of Disentangled Latents (ICML 2025 · similarity 0.87)
- Layer-Wise Representation Fusion for Compositional Generalization (AAAI 2024 · similarity 0.87)
- Disentangled Representation Learning via Modular Compositional Bias (NeurIPS 2025 · similarity 0.87)
- Attention-based Iterative Decomposition for Tensor Product Representation (ICLR 2024 · similarity 0.87)
- Unlocking Compositional Generalization in Continual Few-Shot Learning (NeurIPS 2026 · similarity 0.86)
- In-Context Compositional Learning vis Sparse Coding Transformer (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.