Research map / Probabilistic and scientific modeling / Molecules and proteins
Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Equivariance by Contrast (EbC), an encoder-only contrastive method that jointly learns an embedding and a linear group representation from unlabeled (y, g·y) pairs, with an identifiability proof up to linear maps.
Problem
Learning equivariant embeddings where group actions act as invertible linear maps, using only observation pairs (y, g·y) with unknown group elements and no generative model or group-specific architecture.
Approach
Infers a linear group representation per batch of paired samples via least squares on encoder outputs, applies it to a query embedding, and trains the encoder with a contrastive (InfoNCE-style) objective over negatives; the paper proves identifiability of the latent space and representation up to a linear map under diversity conditions.
Key results
- On synthetic SO(3)/O(3)/GL(3) data, EbC reaches R^2(x) of 99.7-99.8% and R^2(G) of 99.7% for SO(3) and O(3), versus 0% for InfoNCE and about 54-70% for the linear baseline.
- On infinite dSprites, EbC attains Acc(G,5) of 99.91% for the rotation-translation product group, but content accuracy drops to 74.04% versus about 99.97% for InfoNCE.
- Theory shows identifiability of the latent space up to a linear map L and of the group representation up to conjugation by L (rh(g x) = g rh(x)).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace InfoNCE with EbC's learned implicit group representation | Encoder-only representation learning · SO3, O3, and GL3 synthetic data | R²(x) and R²(G): R²(x): 0.0 -> 99.7 (SO3), 0.0 -> 99.8 (O3), 0.1 -> 99.8 (GL3); R²(G): 0.0 -> 99.7 (SO3), 0.0 -> 99.7 (O3), 0.0 -> 99.7 ( | Helped |
| replace InfoNCE with EbC | Representation learning on synthetic group data · SO3, O3, and GL3 synthetic data | Acc(C): 98.9 -> 99.1 (SO3); 99.1 -> 99.2 (O3); 98.5 -> 98.5 (GL3) | No clear effect |
| replace InfoNCE with EbC | Representation learning on image data · Infinite dSprites | Acc(C): 99.97 -> 74.04 | Hurt |
| replace InfoNCE with EbC | Representation learning on image data · Infinite dSprites | Acc(G, 5): 0.36 -> 99.91 | Helped |
| use a symmetric loss (ablation) | EbC · O3 and GL3 synthetic data | Acc(G, 5): O3, negatives both: 89.75 -> 99.83; GL3, negatives both: 99.98 -> 99.93 | Helped |
| use negatives from both input and transformed samples instead of only input samples (ablation) | EbC with non-symmetric loss · GL3 synthetic data | Acc(G, 1): 99.91 -> 93.14 | Hurt |
| use negatives from both input and transformed samples instead of only transformed samples (ablation) | EbC with non-symmetric loss · O3 synthetic data | Acc(G, 5): 95.91 -> 89.75 | Hurt |
| use negatives from both input and transformed samples instead of only transformed samples (ablation) | EbC with non-symmetric loss · GL3 synthetic data | Acc(G, 1): 96.83 -> 93.14 | Hurt |
Surprising findings
- Content classification accuracy on infinite dSprites drops to about 74% for EbC, a trade-off against the invariant content representation of baselines.
- Performance on GL(n) degrades as n grows beyond 5, and GL(9) is not recovered; gradient backpropagation through the least-squares solver recovers n=7.
Limitations
- Content classification accuracy declines when many classes (hundreds or more) are present on idSprites.
- Separation of content and style subspaces requires minimal group dimensionality, which needs costly hyperparameter selection.
- Evaluated only on synthetic data and idSprites; no full-scale real-world image datasets such as 3DIdent; needs about 6-8x d+1 samples in practice.
Experimental setup
Datasets: synthetic group data (SO(n), O(n), GL(n)) infinite dSprites
Benchmarks: infinite dSprites synthetic SO(3), O(3), GL(3) data
Metrics: R^2(x) R^2(G) Acc(G,K) Acc(C,K) TopK accuracy
Baselines: InfoNCE DCL with LDS DCL with SLDS linear baseline
Builds on: InfoNCE CARE Linear dynamical systems contrastive learning (DCL) Nonlinear ICA identifiability SimCLR
Closest papers by meaning
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.90)
- Probing Equivariance and Symmetry Breaking in Convolutional Networks (NeurIPS 2025 · similarity 0.89)
- Neural Fourier Transform: A General Approach to Equivariant Representation Learning (ICLR 2024 · similarity 0.88)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.88)
- Variational Partial Group Convolutions for Input-Aware Partial Equivariance of Rotations and Color-Shifts (ICML 2024 · similarity 0.88)
- Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.