Research map / Detection and segmentation / Open-set and few-shot detection
Learning to Count without Annotations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a reference-based object counter (UnCounTR) with no manual annotations by pasting unsupervised segmented objects onto backgrounds to build "Self-Collages" with pseudo density maps, using a frozen DINO encoder.
Problem
Reference-based (exemplar-conditioned) object counting normally requires costly manually annotated counting data; the paper asks whether counting can be learned without any manual annotations.
Approach
Construct self-supervised training images ("Self-Collages") by pasting self-segmented objects, grouped into unsupervised k-means clusters from DINO features, onto background images; the known pasted positions give free density-map labels and exemplars, and a frozen DINO encoder with a CounTR-style transformer decoder is trained with MSE.
Key results
- On FSC-147 low split, UnCounTR reaches MAE 5.60 vs 6.58 for supervised CounTR and 6.92 for DETR, and RMSE 10.13 vs 48.11 for CounTR.
- On MSO, UnCounTR achieves MAE 1.07 vs 2.34 for supervised CounTR.
- On CARPK, UnCounTR achieves MAE 30.35 vs 19.62 for CounTR, so supervised CounTR remains better there.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with Self-Collages for reference-based counting | UnCounTR · FSC-147 low | MAE: 14.71 -> 5.60 | Helped |
| train with Self-Collages for reference-based counting | UnCounTR · MSO | MAE: 2.34 -> 1.07 | Helped |
| freeze both image encoder Φ and exemplar encoder Ψ (ablation) | UnCounTR · FSC-147 | MAE: 37.64 -> 35.77 | Helped |
| freeze both image encoder Φ and exemplar encoder Ψ (ablation) | UnCounTR · FSC-147 low | MAE: 7.99 -> 5.60 | Helped |
| use segmented pasting and allow overlap (ablation) | UnCounTR Self-Collage construction · FSC-147 | MAE: 36.47 -> 35.77 | Helped |
| use segmented pasting and allow overlap (ablation) | UnCounTR Self-Collage construction · FSC-147 low | MAE: 6.33 -> 5.60 | Helped |
| increase maximum pasted objects from 20 to 50 (ablation) | UnCounTR Self-Collage construction · FSC-147 | MAE: 35.77 -> 30.17 | Helped |
| increase maximum pasted objects from 20 to 50 (ablation) | UnCounTR Self-Collage construction · FSC-147 low | MAE: 5.60 -> 6.65 | Hurt |
| include zero-exemplar training alongside 1–3 exemplars (ablation) | UnCounTR · FSC-147 | MAE: 35.77 -> 36.28 | Hurt |
| include zero-exemplar training alongside 1–3 exemplars (ablation) | UnCounTR · FSC-147 low | MAE: 5.60 -> 6.23 | Hurt |
| replace DINO ViT-B/16 with DINO ViT-B/8 (ablation) | UnCounTR visual encoder · FSC-147 | MAE: 35.77 -> 38.16 | Hurt |
| replace DINO ViT-B/16 with ViT-B/8 pretrained with Leopart (ablation) | UnCounTR visual encoder · FSC-147 | MAE: 35.77 -> 36.04 | No clear effect |
Surprising findings
- Including the zero-shot (salient object) counting task even with a 25% chance lowered performance, attributed to shortcut learning on pasting artefacts.
- Its performance on the FSC-147 high split is weak (MAE 67.17 vs 20.50 for CounTR), indicating limited generalisation to high counts.
Limitations
- Large gap to supervised counters on high-count images (FSC-147 high split), with generalisation limited to training counts of roughly 3-19 objects.
- Performance trails supervised CounTR on CARPK and on the full FSC-147 test set.
- Ambiguous object counts in images are hard to resolve.
Experimental setup
Datasets: ImageNet-1k SUN397 FSC-147 CARPK MSO
Benchmarks: FSC-147 MSO CARPK
Metrics: MAE RMSE Kendall's tau
Baselines: Average baseline Connected components baseline FasterRCNN DETR CounTR LOCA
Builds on: CounTR DINO Leopart FSC-147
Closest papers by meaning
- OmniCount: Multi-label Object Counting with Semantic-Geometric Priors (AAAI 2025 · similarity 0.88)
- T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting (CVPR 2025 · similarity 0.87)
- Point Segment and Count: A Generalized Framework for Object Counting (CVPR 2024 · similarity 0.87)
- CountGD: Multi-Modal Open-World Counting (NeurIPS 2024 · similarity 0.86)
- A Fixed-Point Approach to Unified Prompt-Based Counting (AAAI 2024 · similarity 0.86)
- Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.