Research map / Vision-language and multimodal / Human and embodied modeling
Text-Conditional JEPA for Learning Semantically Rich Visual Representations
Chen Huang, Xianhang Li, Vimal Thilak, Etai Littwin, Joshua M. Susskind
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text conditioning to I-JEPA (ViT-B/16) | I-JEPA · ImageNet-1k linear probing | top-1 accuracy: 72.9 -> 75.8 | Improved |
| add text conditioning to I-JEPA (ViT-L/16) | I-JEPA · ImageNet-1k linear probing | top-1 accuracy: 77.5 -> 79.6 | Improved |
| add text conditioning to I-JEPA (ViT-H/14) | I-JEPA · ImageNet-1k linear probing | top-1 accuracy: 79.3 -> 80.4 | Improved |
| add text conditioning to I-JEPA (ViT-H/14) | I-JEPA · iNat18 linear probing | accuracy: 47.6 -> 54.8 | Improved |
| add text conditioning to I-JEPA (ViT-H/14) | I-JEPA · ADE20k linear segmentation | mIoU: 36.9 -> 39.5 | Improved |
| use MaxPool instead of AvgPool to fuse multi-caption features (ablation) | TC-JEPA ViT-L/16 · ImageNet-1k linear classification | top-1 accuracy: 79.3 -> 79.6 | Improved |
| use MaxPool instead of AvgPool to fuse multi-caption features (ablation) | TC-JEPA ViT-L/16 · ADE20k linear segmentation | mIoU: 38.4 -> 38.8 | Improved |
| use AttentionPool instead of MaxPool for caption fusion (ablation) | TC-JEPA ViT-L/16 · ADE20k linear segmentation | mIoU: 38.8 -> 39.0 | No clear effect |
| add sparse and consistency regularization losses on patch-word similarities (ablation) | TC-JEPA · IN-1k pretraining ablation | downstream performance | Improved |
| increase number of synthetic captions N up to 8 (ablation) | TC-JEPA ViT-L/16 · IN-1k and YFCC15M pretraining | downstream performance | Improved |
| use diverse synthetic captions instead of the single raw caption (N=1) (ablation) | TC-JEPA ViT-L/16 · YFCC15M pretraining | downstream performance | Improved |
| scale loss coefficients lambda and beta by r in [0.5, 2.5] (ablation) | TC-JEPA ViT-L/16 · IN-1k and YFCC15M pretraining | downstream performance | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- SPARC — SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models (CVPR 2025) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning (NeurIPS 2024 · similarity 0.88)
- How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks (NeurIPS 2024 · similarity 0.86)
- Improving fine-grained understanding in image-text pre-training (ICML 2024 · similarity 0.85)
- Denoising with a Joint-Embedding Predictive Architecture (ICLR 2025 · similarity 0.85)
- Retrieval-Enhanced Contrastive Vision-Text Models (ICLR 2024 · similarity 0.85)
- seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
