Research map / Label-efficient and robust learning / Self-supervised and contrastive

T-REGS: Minimum Spanning Tree Regularization for Self-Supervised Learning

NeurIPS 2025 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Data-efficient and robust self-supervision · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes T-REGS, a self-supervised learning regularizer that maximizes the minimum spanning tree length of each branch's embeddings under a soft sphere constraint, shown to prevent dimensional collapse and promote uniformity, and competitive on CIFAR/ImageNet benchmarks.

Problem

Joint-embedding self-supervised learning must avoid dimensional collapse and encourage uniform embedding distributions, but existing contrastive, redundancy-reduction, and asymmetric methods have drawbacks (large batches, second-moment-only constraints, costly optimal transport, lack of theory).

Approach

Regularize each view's batch of embeddings by maximizing the length of its minimum spanning tree (computed with Kruskal's algorithm) combined with a soft sphere constraint; theory shows the optimum for small batches is a regular simplex and that uniform densities maximize the MST-length functional on compact Riemannian manifolds.

Key results

• On CIFAR-10/100 with ResNet-18, BYOL + T-REGS reaches 90.4 / 65.7 top-1 versus 89.5 / 63.7 for BYOL alone, and Barlow Twins + T-REGS reaches 91.8 on CIFAR-10.
• On ImageNet-1k (ResNet-50, 100 epochs), MSE + T-REGS reaches 68.8 top-1, matching the top-tier INTL (69.5) and Zero-CL (68.9) range without auxiliary tuning.
• For CLIP fine-tuning on Flickr30k, L_CLIP + T-REGS reaches 83.2 i-to-t R@1 versus 82.3 for m^3-Mix.

Measured effects reported in the paper

Surprising findings

• Maximizing MST length alone causes divergence or non-convergence; the soft sphere constraint is essential, and the exact sphere radius is reported as not critical.
• A hard normalization constraint on the sphere performed worse than the soft constraint (CIFAR-10 89.2 vs 91.2).

Limitations

• Kruskal's MST has sequential main loop and quadratic-in-batch cost; GPU speed relies on preprocessing parallelization.
• Theoretical guarantees are proved for batch sizes n <= d+1 or asymptotically for large samples; the link to the actual SSL training dynamics is empirical.
• Coefficients beta >= gamma >= lambda were required to avoid collapse and were tuned on ImageNet-1k with 50-epoch probing.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 ImageNet-100 ImageNet-1k Flickr30k MS-COCO

Benchmarks: CIFAR-10 CIFAR-100 ImageNet-100 ImageNet-1k Flickr30k MS-COCO

Metrics: top-1 accuracy R@1 R@5 wall-clock time per step

Baselines: Zero-CL MoCo v2 BYOL Barlow Twins SwAV SimCLR SimSiam W-MSE VICReg CW-RGP INTL SSOLE

Builds on: Barlow Twins BYOL VICReg Zero-CL W-MSE

Closest papers by meaning

• LDReg: Local Dimensionality Regularized Self-Supervised Learning (ICLR 2024 · similarity 0.87)
• Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization (NeurIPS 2024 · similarity 0.85)
• Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections (NeurIPS 2025 · similarity 0.85)
• Graph Geometry-Preserving Autoencoders (ICML 2024 · similarity 0.84)
• SPHERE-JEPA: Spherical Prediction with Homogeneous Embeddings (NeurIPS 2026 · similarity 0.84)
• Finsler Multi-Dimensional Scaling: Manifold Learning for Asymmetric Dimensionality Reduction and Embedding (CVPR 2025 · similarity 0.84)