Research map / Label-efficient and robust learning / Recognition and long-tail
Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations
Yilun Kuang, Yash Dagade, Tim G. J. Rudner, Randall Balestriero, Yann LeCun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| rectify both features and the target distribution (RGN p | z+) instead of other combinations (ablation) | Rectified LpJEPA · CIFAR-100 validation | accuracy vs sparsity tradeoff | Improved |
| match rectified features to an isotropic Gaussian target (ablation) | Rectified LpJEPA · CIFAR-100 validation | accuracy | Hurt |
| do not rectify features or target (LpJEPA) (ablation) | Rectified LpJEPA · CIFAR-100 validation | sparsity | Hurt |
| apply an extra ReLU to LpJEPA features instead of matching the rectified target (ablation) | LpJEPA · linear probe evaluation | accuracy | Hurt |
| vary mean shift mu and p in the target distribution (ablation) | Rectified LpJEPA · pretrained ResNet, ViT, ConvNeXt | empirical L0 norm vs theory | Improved |
| increase sparsity of features up to about 95% zeros (ablation) | Rectified LpJEPA · CIFAR-100 validation | accuracy | No clear effect |
| push sparsity beyond about 95% zeros (mu = -2.5) (ablation) | Rectified LpJEPA · ImageNet-100 linear probe | Acc1 projector: 80.00 -> 67.82 (RGN 2.0(0) vs RGN 2.0(-2.5)) | Hurt |
| use Rectified LpJEPA instead of Gaussian-based LeJEPA | JEPA · ImageNet-100 linear probe (encoder) | Acc1 (%): 84.80 -> 85.08 | Improved |
| use Rectified LpJEPA instead of Non-Negative VICReg | sparse JEPA · ImageNet-100 linear probe (projector) | Acc1 (%): 77.74 -> 80.40 | Improved |
| use Rectified LpJEPA instead of VICReg or NVICReg | JEPA · pairwise independence | normalized HSIC | Improved |
| use eigenvectors of feature covariance as projection vectors (ablation) | RDMReg · convergence speed | convergence to optimal performance | Improved |
| raise mean shift to reduce sparsity, trading sparsity for entropy (ablation) | Rectified LpJEPA · d(xi)-dimensional entropy | entropy | Mixed |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- VICReg — VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning (ICLR 2022) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Denoising with a Joint-Embedding Predictive Architecture (ICLR 2025 · similarity 0.85)
- Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures (NeurIPS 2026 · similarity 0.84)
- Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning (NeurIPS 2024 · similarity 0.84)
- From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit (NeurIPS 2025 · similarity 0.83)
- PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment (NeurIPS 2026 · similarity 0.83)
- Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
