Research map / Learning theory and algorithms / Kernels and regression theory

SPHERE-JEPA: Spherical Prediction with Homogeneous Embeddings

NeurIPS 2026 · Learning theory and algorithms › Kernels and regression theory · approach: RKHS kernel methods · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Extends LeJEPA's minimax optimality analysis to Riemannian manifolds, showing the uniform hyperspherical distribution is optimal for k-NN and exponential-kernel ridge regression, and proposes SPHERE-JEPA with a spherical Cramér–Wold regularizer (SUSReg).

Problem

Characterizing the optimal geometry of SSL representations when embeddings lie on lower-dimensional manifolds such as the hypersphere, rather than in Euclidean space as assumed by LeJEPA.

Approach

Under a worst-case minimax formulation, uniform distributions on manifolds (and the uniform sphere for exponential-kernel KRR) minimize the bias of k-NN and KRR; SUSReg enforces this by matching 1D projections of normalized embeddings to the spherical marginal density via the Epps–Pulley test, combined with an MSE invariance loss on normalized embeddings.

Key results

• On ImageNet-1K with ViT-B/14 (100 epochs, no EMA), SPHERE-JEPA reaches 61.0% linear probe vs 59.2% for LeJEPA with equal k-NN (50.4%).
• On a texture retrieval task, SPHERE-JEPA reaches 92.9 mAP vs 85.6 for LeJEPA (averaged over four texture datasets).
• On ImageNet-100 with ResNet-18 (4 seeds), LeJEPA outperforms SPHERE-JEPA on linear probe (72.46 vs 71.22) and k-NN (67.61 vs 65.67).

Surprising findings

• On ImageNet-100 and Galaxy10, LeJEPA outperforms SPHERE-JEPA on linear and k-NN probing, contrary to the theory-motivated expectation.
• Normalizing LeJEPA's Gaussian projection-head outputs does not give equivalent objectives, since Gaussian matching penalizes radial mismatch.

Limitations

• Empirical validation limited: large-scale ViT-B/14 result is a single run without hyperparameter tuning.
• SPHERE-JEPA underperforms LeJEPA on ImageNet-100 and Galaxy10 benchmarks.
• Optimality guarantees cover only the stated estimators and target classes; not finite-width MLP feature learning or fine-tuning.

Experimental setup

Datasets: ImageNet-1K ImageNet-100 Galaxy10 ADE20K DTD Aircraft CIFAR10 CIFAR100 Flowers Food Pets procedural textures (Disk, Cloud, Flake, Wood)

Benchmarks: ImageNet-1K ImageNet-100 Galaxy10 ADE20K DTD Aircraft CIFAR10 CIFAR100 Flowers Food Pets procedural texture retrieval

Metrics: linear probe accuracy k-NN accuracy mIoU pixel accuracy Recall@K mAP

Baselines: LeJEPA VICReg InfoNCE BYOL

Builds on: LeJEPA SIGReg Cramér–Wold theorem Epps–Pulley test DINO

Closest papers by meaning

• Optimization Can Learn Johnson Lindenstrauss Embeddings (NeurIPS 2024 · similarity 0.84)
• T-REGS: Minimum Spanning Tree Regularization for Self-Supervised Learning (NeurIPS 2025 · similarity 0.84)
• Intrinsic Riemannian Cross-covariance for Manifold-valued Random Objects (NeurIPS 2026 · similarity 0.84)
• Harmonizing Geometry and Uncertainty: Diffusion with Hyperspheres (ICML 2025 · similarity 0.84)
• Wrapped Gaussian on the manifold of Symmetric Positive Definite Matrices (ICML 2025 · similarity 0.84)
• Euclidean Embedding of Data Using Local Distances (NeurIPS 2026 · similarity 0.84)