Research map / Detection and segmentation / Open-set and few-shot detection

OmniCount: Multi-label Object Counting with Semantic-Geometric Priors

AAAI 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Object-level representation learning · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes OmniCount, a training-free open-vocabulary method that counts multiple object categories in one pass by combining SAN semantic masks, Marigold depth priors, and reference-point-guided SAM, and introduces the OmniCount-191 multi-label counting benchmark.

Problem

Object counting is usually class-specific or class-agnostic with exemplars, requiring a separate pass per category; existing multi-label counters are trained and fail on non-atomic objects and occlusion.

Approach

Combine semantic priors (SAN open-vocabulary segmentation) with geometric priors (Marigold depth) to refine masks via kNN-based recovery, derive per-class reference points from feature activations, and prompt SAM to produce instance masks whose count gives each category's number, all without training.

Key results

• On OmniCount-191 multi-label text-prompt counting, OmniCount reaches mRMSE 0.70 and mRMSE-nz 2.00, versus 0.50 / 1.87 for the trained GrREC (best learning-based) and 0.95 / 2.89 for TFOC.
• On FSC-147 single-label counting with box prompts, OmniCount achieves MAE 18.63 vs 19.95 for TFOC (training-free), while trained BMNet+ reaches 14.62.
• Ablation on OmniCount-191 shows that removing geometric priors raises m-RMSE from 0.70 to 2.12, and removing reference points raises it to 1.62 with depth but without RP.

Measured effects reported in the paper

Surprising findings

• Directly counting reference points without SAM gives mRMSE 0.98 versus 0.70 with SAM, and the paper notes that vanilla SAM's uniform point grid causes overcounting.
• On FSC-147 text prompts, OmniCount's MAE (21.46) is worse than GrREC's (10.12) even though its SRE is much lower.

Limitations

• Struggles with low-resolution and low-illumination images, and with objects far from the camera.
• Fails in extremely dense crowd counting scenarios where training-based density methods are needed.
• Cannot directly take referring expressions; relies on class-name semantic segmentation.

Experimental setup

Datasets: OmniCount-191 PASCAL VOC FSC-147 CARPK REC-8K

Benchmarks: OmniCount-191 PASCAL VOC FSC-147 CARPK REC-8K VQA-v2

Metrics: mRMSE mRMSE-nz MAE RMSE NAE SRE Precision Recall F1 mAP

Baselines: ILC GrREC Grounding-DINO CLIPSeg TFOC PSeCo ZSOC Vanilla SAM BMNet+ CounTX

Builds on: Segment Anything Model Side Adapter Network Marigold TFOC GrREC

Closest papers by meaning

• CountGD: Multi-Modal Open-World Counting (NeurIPS 2024 · similarity 0.91)
• T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting (CVPR 2025 · similarity 0.88)
• VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting (AAAI 2024 · similarity 0.88)
• Learning to Count without Annotations (CVPR 2024 · similarity 0.88)
• DAVE - A Detect-and-Verify Paradigm for Low-Shot Counting (CVPR 2024 · similarity 0.87)
• COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts (CVPR 2025 · similarity 0.87)