Research map / Vision-language and multimodal / CLIP and zero-shot
T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes T2ICount, a single-step Stable Diffusion-based zero-shot counter with a hierarchical semantic correction module and a diffusion-attention-guided loss, plus a re-annotated harder FSC-147-S subset for text-guided counting evaluation.
Problem
Text-guided zero-shot object counting models based on CLIP show weak sensitivity to text prompts, often counting the majority object rather than the text-specified category, and FSC-147 benchmark annotation bias hides this failure.
Approach
Uses single-step features from a pretrained text-to-image diffusion U-Net (Stable Diffusion v1.5) for counting, adds a Hierarchical Semantic Correction Module to refine text-image alignment across scales, and supervises intermediate similarity maps with a loss whose positive/negative/ambiguous labels come from diffusion cross-attention maps.
Key results
- On FSC-147 test set, T2ICount achieves MAE 11.76 and RMSE 97.86, reducing MAE by 25.9% relative to CounTX (15.88).
- On the re-annotated FSC-147-S subset, T2ICount reaches MAE 4.69 and RMSE 8.06, versus 31.30 / 98.80 for CounTX.
- Ablation on FSC-147-S shows LRRC cuts baseline MAE from 24.34 to 9.59, and HSCM further reduces it to 4.69.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use T2ICount instead of the strongest reported zero-shot baseline | Zero-shot object counting · FSC-147 test | MAE: 13.30 -> 11.76 | Helped |
| Use T2ICount instead of the strongest reported zero-shot baseline | Zero-shot object counting · FSC-147 test | RMSE: 103.42 -> 97.86 | Helped |
| Use T2ICount instead of the strongest reported baseline | Zero-shot object counting · FSC-147-S | MAE: 39.01 -> 4.69 | Helped |
| Use T2ICount instead of the strongest reported baseline | Zero-shot object counting · FSC-147-S | RMSE: 61.34 -> 8.06 | Helped |
| Use T2ICount instead of CounTX | Zero-shot object counting · CARPK | MAE: 8.13 -> 8.61 | Hurt |
| Use T2ICount instead of CounTX | Zero-shot object counting · CARPK | RMSE: 10.87 -> 13.47 | Hurt |
| Add LRRC to the baseline (ablation) | T2ICount · FSC-147-S | MAE: 24.34 -> 9.59 | Helped |
| Add LRRC to the baseline (ablation) | T2ICount · FSC-147-S | RMSE: 64.74 -> 22.25 | Helped |
| Add LRRC to the baseline (ablation) | T2ICount · FSC-147 test | MAE: 14.66 -> 14.55 | Helped |
| Add LRRC to the baseline (ablation) | T2ICount · FSC-147 test | RMSE: 111.62 -> 106.21 | Helped |
| Add HSCM to the LRRC-equipped baseline (ablation) | T2ICount · FSC-147 test | MAE: 14.55 -> 11.76 | Helped |
| Add HSCM to the LRRC-equipped baseline (ablation) | T2ICount · FSC-147-S | MAE: 9.59 -> 4.69 | Helped |
Surprising findings
- Existing zero-shot counters achieve competitive FSC-147 scores while failing badly on FSC-147-S, suggesting benchmark majority-class bias masks text insensitivity.
Limitations
- Text sensitivity is weakened by single-step denoising, requiring the HSCM and LRRC to compensate.
- FSC-147-S is a small subset of 196 images, limiting evaluation scale.
- Authors note future work should build a more diverse dataset with richer object categories.
Experimental setup
Datasets: FSC-147 CARPK
Benchmarks: FSC-147 FSC-147-S CARPK
Metrics: MAE RMSE
Baselines: CLIP-Count CounTX VLCounter PseCo DAVE VA-Count GeCo LOCA
Builds on: Stable Diffusion CounTX CLIP-Count VLCounter CUT
Closest papers by meaning
- VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting (AAAI 2024 · similarity 0.92)
- CountGD: Multi-Modal Open-World Counting (NeurIPS 2024 · similarity 0.89)
- Single Domain Generalization for Few-Shot Counting via Universal Representation Matching (CVPR 2025 · similarity 0.88)
- Make It Count: Text-to-Image Generation with an Accurate Number of Objects (CVPR 2025 · similarity 0.88)
- OmniCount: Multi-label Object Counting with Semantic-Geometric Priors (AAAI 2025 · similarity 0.88)
- DAVE - A Detect-and-Verify Paradigm for Low-Shot Counting (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.