Research map / Detection and segmentation / CNN features
PBECount: Prompt-Before-Extract Paradigm for Class-Agnostic Counting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the prompt-before-extract paradigm and a pure CNN (ResNet34 U-Net) counter, PBECount, that conditions on exemplar probability maps and uses a dynamic threshold, peak-aware MSE loss, and loop optimization to count only exemplar-similar objects.
Problem
Class-agnostic counting in multi-class scenes: existing extract-then-match methods count all objects rather than only those similar to the exemplars, and the extract-and-match ViT paradigm overly relies on the ViT architecture.
Approach
Feed exemplar probability maps concatenated with the query image into a CNN backbone before feature extraction (prompt-before-extract), generalizing extract-and-match as a special case; add a score head that predicts a per-image dynamic threshold for NMS-filtered probability-map detections.
Key results
- PBECount reports MAE 8.88 on FSC-147 validation and 7.71 on test, the latter a 10.97% reduction over DAVE.
- On the FSC-147 Mul multi-class subset, PBECount reaches MAE 4.39 (val) and 4.86 (test), outperforming CACViT (6.00 and 5.74).
- On cross-dataset CARPK, PBECount achieves MAE 6.27 and RMSE 9.08, a 24.46% MAE reduction over CACViT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use prompt-before-extract PBECount instead of DAVE | Class-agnostic counting · FSC-147 Test | MAE: 8.66 -> 7.71; 10.97% reduction | Helped |
| use PBECount instead of CACViT | Cross-dataset class-agnostic counting · CARPK | MAE: 24.46% reduction | Helped |
| add count result normalization (ablation) | PBECount · FSC-147 | MAE: 9.55% | Helped |
| add loop optimization (ablation) | PBECount · FSC-147 | MAE: 9.23% | Helped |
| add exemplar size normalization (ablation) | PBECount · FSC-147 | MAE: 30.27% | Helped |
| use peak-aware MSE loss instead of regular MSE loss (ablation) | PBECount · FSC-147 | MAE: 13.84% | Helped |
| use dynamic threshold instead of a fixed threshold (ablation) | PBECount · FSC-147 | MAE: 16.31% | Helped |
Surprising findings
- Disabling exemplar size normalization at test time caused the largest performance drop (30.27% relative MAE increase), larger than removing any architectural component.
Limitations
- Test RMSE on FSC-147 (44.92) is worse than CACViT and DAVE, indicating weaker performance on some metrics.
- Several test-time heuristics (loop optimization, count normalization, size normalization) are hand-designed and add complexity.
Experimental setup
Datasets: FSC-147 CARPK
Benchmarks: FSC-147 FSC-147 Mul CARPK
Metrics: MAE RMSE
Baselines: FamNet BMNet+ SAFECount CounTR LOCA CACViT DAVE
Builds on: CACViT extract-and-match CounTR Tracking objects as points (probability map input) DAVE detect-and-verify
Closest papers by meaning
- Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting (AAAI 2024 · similarity 0.92)
- A Fixed-Point Approach to Unified Prompt-Based Counting (AAAI 2024 · similarity 0.87)
- Point Segment and Count: A Generalized Framework for Object Counting (CVPR 2024 · similarity 0.86)
- VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting (AAAI 2024 · similarity 0.84)
- Learning to Count without Annotations (CVPR 2024 · similarity 0.83)
- Referring Expression Counting (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.