Research map / Detection and segmentation / Open-set and few-shot detection
Referring Expression Counting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Referring Expression Counting (REC), a task that counts objects by attribute within a class, with the REC-8K dataset and GroundingREC, a GroundingDINO-based counter with global-local feature fusion and contrastive attribute learning.
Problem
Existing counting tasks (class-specific, class-agnostic few-shot and zero-shot) count at class level and cannot distinguish objects of the same class by fine-grained attributes specified in referring expressions.
Approach
Reframe counting as referring expression grounding: adapt an open-set detector (GroundingDINO) to output point predictions for attribute-conditioned targets, add a global-local feature fusion module for relational attributes, and a contrastive loss that separates attributes of the same class.
Key results
- On REC-8K test, GroundingREC achieves MAE 6.50 and RMSE 19.79 with F1 0.69, versus 8.88 MAE for finetuned GroundingDINO and 11.84 MAE for CounTX.
- On FSC-147 zero-shot test, GroundingREC reaches MAE 10.12, beating CounTX (15.88) and other zero-shot methods, and is comparable to few-shot methods such as CACViT (9.13).
- Ablation on REC-8K shows removing contrastive learning raises test RMSE from 19.79 to 21.00 and removing global-local fusion raises test MAE from 6.50 to 7.33.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GroundingREC instead of fine-tuned GroundingDino | REC-8K · REC-8K test | MAE: 8.88 -> 6.50 | Helped |
| use GroundingREC instead of fine-tuned GroundingDino | REC-8K · REC-8K test | RMSE: 21.95 -> 19.79 | Helped |
| use GroundingREC instead of fine-tuned GroundingDino | REC-8K · REC-8K test | F1: 0.66 -> 0.69 | Helped |
| use GroundingREC instead of fine-tuned GroundingDino | FSC-147 zero-shot · FSC-147 test zero-shot | MAE: 10.82 -> 10.12 | Helped |
| use GroundingREC instead of CounTX | FSC-147 zero-shot · FSC-147 test zero-shot | RMSE: 106.29 -> 107.19 | Hurt |
| add global-local feature fusion (ablation) | GroundingREC · REC-8K val | MAE: 7.31 -> 6.80 | Helped |
| add global-local feature fusion (ablation) | GroundingREC · REC-8K test | MAE: 7.33 -> 6.50 | Helped |
| add global-local feature fusion (ablation) | GroundingREC · REC-8K test | RMSE: 19.71 -> 19.79 | Hurt |
| add global-local feature fusion (ablation) | GroundingREC · REC-8K test | F1: 0.67 -> 0.69 | Helped |
| add contrastive learning (ablation) | GroundingREC · REC-8K val | MAE: 7.25 -> 6.80 | Helped |
| add contrastive learning (ablation) | GroundingREC · REC-8K val | RMSE: 15.97 -> 18.13 | Hurt |
| add contrastive learning (ablation) | GroundingREC · REC-8K test | RMSE: 21.00 -> 19.79 | Helped |
Surprising findings
- Pretrained GroundingDINO without finetuning has low localization F1 (0.34) despite reasonable counting error, showing counting error can hide FP/FN cancellation.
Limitations
- Fails on complex location, negation, ambiguous and text-involving attributes.
- Misleading elements such as reflections and building-facade humans cause false positives.
- Relies on text reasoning and image-text alignment that remain open problems.
Experimental setup
Datasets: REC-8K FSC-147 JHU Crowd NWPU VisDrone DETRAC Carpk Mall Crowd Surveillance
Benchmarks: REC-8K FSC-147
Metrics: MAE RMSE Precision Recall F1
Baselines: ZSC TFOC CounTX GroundingDINO FamNet BMNet+ CounTR CACViT
Builds on: GroundingDINO DETR FSC-147 CounTX
Closest papers by meaning
- Exploring Contextual Attribute Density in Referring Expression Counting (CVPR 2025 · similarity 0.87)
- Point Segment and Count: A Generalized Framework for Object Counting (CVPR 2024 · similarity 0.86)
- CountGD: Multi-Modal Open-World Counting (NeurIPS 2024 · similarity 0.85)
- Learning to Count without Annotations (CVPR 2024 · similarity 0.85)
- VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting (AAAI 2024 · similarity 0.85)
- A Fixed-Point Approach to Unified Prompt-Based Counting (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.