Research map / Label-efficient and robust learning / Recognition and long-tail
A Fixed-Point Approach to Unified Prompt-Based Counting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a unified class-agnostic counting framework that turns box, point, and text prompts into prompt masks, trained with a fixed-point loss and contrastive training to reduce dataset bias.
Problem
Class-agnostic counting models rely on a single prompt type (usually boxes), and FSC-147 training bias causes models to count salient objects rather than the prompted ones.
Approach
Convert box, point, and text prompts into a shared prompt mask (text via MaskCLIP-style CLIP value embeddings with softmax over an image-specific concept dictionary), refine density maps via a fixed-point recurrent loop trained with implicit differentiation, and use contrastive image pairs with all-zero negatives.
Key results
- Box prompt on FSC-147 test: MAE 16.68 / MSE 105.08 versus CounTR 14.12 / 108.81 and TFPOC 19.95 / 132.16 without scale prior.
- Fixed-point loss reduces validation MAE from 19.09 (L2) to 16.98.
- Cross-dataset CARPK box prompt MAE 7.83 vs BMNet 10.44 and TFPOC 10.97.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use softmax prompt masks instead of cosine masks (ablation) | unified prompt counting model · FSC-147 validation, box prompts | MAE: 18.55 -> 16.87 | Helped |
| use softmax prompt masks instead of cosine masks (ablation) | unified prompt counting model · FSC-147 validation, text prompts | MAE: 20.59 -> 16.92 | Helped |
| use softmax prompt masks instead of cosine masks (ablation) | unified prompt counting model · FSC-147 validation, point prompts | MAE: 18.46 -> 17.16 | Helped |
| add fixed-point loss instead of vanilla L2 loss (ablation) | unified prompt counting model · FSC-147 validation | MAE: 19.09 -> 16.98 | Helped |
| add fixed-point loss instead of vanilla L2 loss (ablation) | unified prompt counting model · FSC-147 test | MAE: 17.22 -> 16.45 | Helped |
| add contrastive training (ablation) | unified prompt counting model · FSC-147 validation negative samples | N-MAE: 38.09 -> 1.90 | Helped |
| add contrastive training (ablation) | unified prompt counting model · FSC-147 test negative samples | N-MAE: 48.77 -> 3.89 | Helped |
| add scale prior | unified prompt counting model · FSC-147 validation, box prompts | MAE: 16.87 -> 12.80 | Helped |
| add scale prior | unified prompt counting model · FSC-147 test, box prompts | MAE: 16.68 -> 11.86 | Helped |
| use only the infinity loss, omitting the finite loss (ablation) | fixed-point training · FSC-147 test | MAE: 26.97 | Hurt |
| combine infinity loss with alternative finite loss L_T' instead of L_T (ablation) | fixed-point training · FSC-147 test | MSE: 104.72 vs. 105.91 | Hurt |
| set refinement iterations to T=2 (ablation) | fixed-point inference · FSC-147 | MAE | Helped |
Surprising findings
- Text prompt mask via cosine similarity alone performs worse than softmax over a concept dictionary, and noisy cosine masks also degrade box and point tracks.
- The L-infinity-only fixed-point loss (no finite-step term) performs much worse (MAE 26.97 on test), worse than TFPOC.
Limitations
- Scale prior variants are needed to beat CounTR; the unified model does not win on box test MAE without scale prior.
- Text prompt mask depends on an external captioning model (LLaMA-Adapter V2) and Spacy to build concept dictionaries.
Experimental setup
Datasets: FSC-147 CARPK
Benchmarks: FSC-147 CARPK
Metrics: MAE MSE N-MAE N-MSE
Baselines: GMN FamNet BMNet SPDCN CounTR TFPOC ZSOC CLIP-C
Builds on: MaskCLIP CounTR CLIP implicit differentiation
Closest papers by meaning
- Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting (AAAI 2024 · similarity 0.87)
- PBECount: Prompt-Before-Extract Paradigm for Class-Agnostic Counting (AAAI 2025 · similarity 0.87)
- Learning to Count without Annotations (CVPR 2024 · similarity 0.86)
- Referring Expression Counting (CVPR 2024 · similarity 0.85)
- CountGD: Multi-Modal Open-World Counting (NeurIPS 2024 · similarity 0.84)
- OmniCount: Multi-label Object Counting with Semantic-Geometric Priors (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.