Research map / 3D vision / Face and shape
ShapeEmbed: a self-supervised learning framework for 2D contour quantification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ShapeEmbed, a VAE that encodes normalized Euclidean distance matrices of 2D contours with circular-padded ResNet-18 encoder and indexation-invariant reconstruction loss to get shape descriptors invariant to translation, scale, rotation, reflection and indexing.
Problem
Quantifying 2D object shape with descriptors that remain invariant to size, orientation, position and contour point indexing, especially for biological imaging where objects are unaligned and size varies.
Approach
Encode distance matrices of closed contours (inherently translation/rotation invariant) with a VAE whose encoder uses circular padding and reflection-symmetric summation to be indexation/reflection invariant, paired with a min-over-all-2N-reindexings reconstruction loss and Euclidean distance matrix regularizers.
Key results
- ShapeEmbed reaches F1 0.96 ± 0.01 on MNIST and 0.75 ± 0.02 on MPEG-7, above MAE ViT-B (0.95 and 0.65) and region properties (0.81 and 0.70).
- On biological data, ShapeEmbed+Sz reaches F1 0.76 on MEF and 0.87 on BBBC010, versus 0.72 and 0.82 for region properties.
- On rotated/translated rMNIST, ShapeEmbed scores F1 0.85 versus 0.66 for O2VAE and 0.38 for a vanilla VAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ShapeEmbed descriptors instead of the strongest listed baseline | Shape classification · MNIST | F1-score: 0.95 \pm 0.03 -> 0.96 \pm 0.01 | Helped |
| Use ShapeEmbed descriptors instead of Region prop. | Shape classification · MPEG-7 | F1-score: 0.70 \pm 0.01 -> 0.75 \pm 0.02 | Helped |
| Add object size as an extra feature | ShapeEmbed · MEF | F1-score: 0.67 \pm 0.01 -> 0.76 \pm 0.01 | Helped |
| Add object size as an extra feature | ShapeEmbed · BBBC010 | F1-score: 0.83 \pm 0.00 -> 0.87 \pm 0.01 | Helped |
| Add indexation invariance using circular padding and indexation-invariant reconstruction loss (ablation) | ShapeEmbed · sMNIST | F1-score: 0.88 \pm 0.01 -> 0.95 \pm 0.00 | Helped |
| Add indexation invariance using circular padding and indexation-invariant reconstruction loss (ablation) | ShapeEmbed · sMPEG-7 | F1-score: 0.59 \pm 0.07 -> 0.70 \pm 0.09 | Helped |
| Normalize distance matrices for scale invariance (ablation) | ShapeEmbed · sMNIST | F1-score: 0.91 \pm 0.01 -> 0.95 \pm 0.00 | Helped |
| Normalize distance matrices for scale invariance (ablation) | ShapeEmbed · sMPEG-7 | F1-score: 0.42 \pm 0.02 -> 0.70 \pm 0.09 | Helped |
| Use distance matrices instead of binary masks as VAE input (ablation) | ShapeEmbed · rMNIST | F1-score: 0.38 \pm 0.01 -> 0.85 \pm 0.01 | Helped |
| Use distance matrices instead of binary masks as VAE input (ablation) | ShapeEmbed · rMPEG-7 | F1-score: 0.04 \pm 0.02 -> 0.66 \pm 0.05 | Helped |
| Include symmetry regularization (ablation) | ShapeEmbed · MNIST | F1-score: 0.941 \pm 0.006 -> 0.963 \pm 0.007 | Helped |
| Include diagonal regularization (ablation) | ShapeEmbed · MNIST | F1-score: 0.952 \pm 0.005 -> 0.963 \pm 0.007 | Helped |
Surprising findings
- Region properties (classical statistics) outperform ShapeEmbed on MEF without size information, attributed to experimentally induced size differences; summary statistics perform exceptionally well on MEF.
- MAE ViT-B achieves lower log-loss (0.062) than ShapeEmbed (0.187) on MNIST despite lower F1 margin.
Limitations
- Requires simply-connected contours; objects with holes are not handled directly.
- Limited to 2D images; 3D extension left for future work.
- Reconstruction from MDS is non-unique and regularization terms add little to classification performance.
Experimental setup
Datasets: MNIST MPEG-7 CE-Shape-1 Part B BBBC010 MEF
Benchmarks: MNIST MPEG-7 BBBC010 MEF sMNIST sMPEG-7 rMNIST rMPEG-7
Metrics: F1-score accuracy precision recall log-loss
Baselines: Region properties Elliptical Fourier Descriptors SimCLR MAE (ViT-B/L/H) O2VAE Vanilla VAE
Builds on: O2VAE ResNet-18 SimCLR Masked Autoencoder Elliptical Fourier Descriptors
Closest papers by meaning
- Symmetric Shape-Preserving Autoencoder for Unsupervised Real Scene Point Cloud Completion (CVPR 2023 · similarity 0.86)
- Straight to Shapes: Real-Time Detection of Encoded Shapes (CVPR 2017 · similarity 0.86)
- $\texttt{NAISR}$: A 3D Neural Additive Model for Interpretable Shape Representation (ICLR 2024 · similarity 0.86)
- Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models (NeurIPS 2025 · similarity 0.86)
- ShapeMatcher: Self-Supervised Joint Shape Canonicalization Segmentation Retrieval and Deformation (CVPR 2024 · similarity 0.85)
- Learning Partonomic 3D Reconstruction from Image Collections (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.