Research map / Vision-language and multimodal / CLIP and zero-shot
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CLIP (InfoNCE, no Hopfield) with CLOOB | ResNet-50 pretrained on Conceptual Captions · ImageNet zero-shot | top-1 accuracy: 20.33 -> 23.97 | Helped |
| Replace CLIP with CLOOB | ResNet-50 pretrained on Conceptual Captions · GTSRB zero-shot | accuracy: 7.66 -> 6.38 | No clear effect |
| Replace CLIP reimplementation with CLOOB | ResNet-50 pretrained on YFCC · 8 zero-shot datasets | accuracy: ImageNet 34.6 -> 35.7; UCF101 27.2 -> 25.3 | Mixed |
| Replace CLIP reimplementation with CLOOB using larger encoders | ResNet-101 and ResNet-50x4 on YFCC · ImageNet zero-shot | accuracy: 35.3 -> 37.1 (RN-101); 37.7 -> 39.0 (RN-50x4) | Helped |
| Replace OpenAI CLIP with CLOOB for linear probing | ResNet-50 on YFCC · Country211 linear probing | accuracy: 23.1 -> 20.6 | Hurt |
| Add modern Hopfield network retrieval to the contrastive model (ablation) | CLOOB · embedding covariance spectrum | number of effective eigenvalues | Helped |
| Use only InfoLOOB without Hopfield networks (ablation) | CLOOB · zero-shot downstream tasks | zero-shot accuracy | Hurt |
| Combine Hopfield networks with InfoLOOB (vs either alone) (ablation) | CLOOB · zero-shot downstream tasks | zero-shot accuracy | Helped |
| Make temperature learnable with InfoLOOB instead of fixed tau^-1 = 30 (ablation) | CLOOB · training behavior | training stability | Hurt |
Closest papers by meaning
- Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge (NeurIPS 2024 · similarity 0.87)
- Demystifying CLIP Data (ICLR 2024 · similarity 0.87)
- Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP (ICLR 2024 · similarity 0.87)
- A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation (ICLR 2024 · similarity 0.86)
- Meta CLIP 2: A Worldwide Scaling Recipe (NeurIPS 2025 · similarity 0.86)
- Diffusion Feedback Helps CLIP See Better (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.