Research map / Vision-language and multimodal / CLIP and zero-shot
Teaching CLIP to Count to Ten
Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, Tali Dekel
This paper tried a common change: Add a contrastive loss
add counting contrastive loss with counterfactual captions on a filtered counting set · CountBench · accuracy: 31.67 -> 75.93 Improved
What 1,317 accepted papers report for this change
- 1,253 improved
- 22 no clear effect
- 16 mixed
- 26 hurt
- Beyond Looking Up, Try Looking Around: Harmonizing Global Structure and Local Consistency in Optimal Transport for Short Text Clustering ICML 2026 · add contrastive learning · SearchSnippets · ACC: 56.65 -> 80.12Improved
- SleepLM: Natural-Language Intelligence for Human Sleep ICML 2026 · add contrastive objective to captioning (CoCa vs Cap) · sleep event IoU · IoU: 29.2 -> 27.4Hurt
- LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations ICML 2026 · add contrastive loss L_con · FS1000 average · Spearman correlation: 0.759 -> 0.789Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add counting contrastive loss with counterfactual captions on a filtered counting set | CLIP-B/32 · CountBench | accuracy: 31.67 -> 75.93 | Improved |
| add counting contrastive loss with counterfactual captions on a filtered counting set | BASIC-S · CountBench | accuracy: 17.97 -> 69.02 | Improved |
| add counting loss L_count on top of finetuning on the filtered counting set (ablation) | CLIP-B/32 · CountBench | accuracy: 44.26 -> 75.93 | Improved |
| use restrictive detector-based count filtering instead of naive number filtering (ablation) | CLIP-B/32 · CountBench | accuracy: 49.81 -> 75.93 | Improved |
| use restrictive filtering instead of naive filtering (ablation) | BASIC-S · CountBench | accuracy: 28.68 -> 69.02 | Improved |
| finetune on filtered counting set without counting loss (ablation) | BASIC-S · CountBench | accuracy: 22.75 -> 30.59 | Improved |
| apply counting finetuning to CLIP-B/32 | CLIP-B/32 · zero-shot classification benchmarks | accuracy: ImageNet 62.93 -> 64.06; EuroSAT 45.23 -> 37.69 | Mixed |
| increase counting fraction of batch p from 1/32 to 1/8 (ablation) | CLIP-B/32 · CountBench | accuracy: 75.93 -> 70.19 | Hurt |
| increase counting loss weight lambda from 1 to 5 or 10 (ablation) | CLIP-B/32 · CountBench | accuracy: 75.93 -> 73.15 (lambda 5), 72.59 (lambda 10) | Hurt |
| lower counting loss weight lambda to 0.1 (ablation) | CLIP-B/32 · CountBench | accuracy: 75.93 -> 69.44 | Hurt |
| condition Imagen on counting-aware CLIP instead of baseline CLIP | Imagen · DrawBench counting | accuracy: 24.12 -> 40.35 | Improved |
| condition Imagen on counting-aware CLIP for numbers 2-10 | Imagen · CIFAR10 prompts 2-10 | accuracy: 20.00 -> 50.18 | Improved |
Closest papers by meaning
- VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting (AAAI 2024 · similarity 0.92)
- T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting (CVPR 2025 · similarity 0.90)
- CountGD: Multi-Modal Open-World Counting (NeurIPS 2024 · similarity 0.89)
- Understanding Counting Mechanisms in Large Language and Vision-Language Models (CVPR 2026 · similarity 0.89)
- UNICBench: UNIfied Counting Benchmark for MLLM (CVPR 2026 · similarity 0.88)
- CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
