Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Single Domain Generalization for Few-Shot Counting via Universal Representation Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes URM, a single-domain generalization few-shot counting model that distills universal vision-language prototypes from frozen CLIP into the exemplar-to-image matching process, evaluated on FSC-147 and FSCD-LVIS cross-domain and in-domain settings.
Problem
Few-shot counting models that follow an extract-then-match pipeline degrade severely under domain shift when trained on a single narrow source domain.
Approach
Replace narrow learned prototypes with universal vision and language prototypes distilled from frozen CLIP (MaskCLIP-style mask-pooled visual features and CuPL-style LLM-generated text prompts via feature mimicry), and match these prototypes with image features via cross attention; distillation is used only during training.
Key results
- Cross-domain few-shot FSC-147 to FSCD-LVIS: URM achieves MAE 21.87 and RMSE 38.42, versus 25.11/41.32 for MPCount (prior best DG method).
- Reverse direction FSCD-LVIS to FSC-147 few-shot: URM MAE 21.17, RMSE 73.42, improving over MPCount's 22.07/80.17.
- In-domain FSC-147 test set: URM reports MAE 10.15 and RMSE 54.83, while URM+DAVE reaches 8.32 MAE, outperforming LOCA and DAVE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill universal vision-language knowledge into prototypes (ablation) | few-shot counting model · FSC-147 → FSCD-LVIS few-shot | MAE: 30.15 -> 21.87 | Helped |
| use a naive language template for language distillation (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 30.15 -> 25.44 | Helped |
| add generated customized language prompts (ablation) | URM with language distillation · FSC-147 → FSCD-LVIS few-shot | MAE: 25.44 -> 23.83 | Helped |
| use global pooling instead of the [CLS] token for vision distillation (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 23.53 -> 22.94 | Helped |
| use mask pooling for locality representation and noise removal (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 22.94 -> 21.87 | Helped |
| balance vision and language distillation with α=0.9 rather than using only one source (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 26.54 -> 21.87 | Helped |
| use three prototype-learning and three matching attention layers (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 22.77 -> 21.87 | Helped |
| use a larger CLIP ViT-L/14 teacher instead of ViT-B/16 (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 21.87 -> 22.63 | Hurt |
| use SigLIP ViT-B/16 instead of OpenAI CLIP ViT-B/16 (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 21.87 -> 21.33 | Helped |
| add the DAVE detect-and-verify paradigm (ablation) | URM · FSC-147 test | MAE: 10.15 -> 8.32 | Helped |
| use URM rather than MPCount | few-shot counting · FSC-147 → FSCD-LVIS few-shot | MAE: 25.11 -> 21.87 | Helped |
| generate prompts with GPT4 when category names are not given (ablation) | URM · FSC-147 → FSCD-LVIS few-shot | MAE: 22.45 | Helped |
Surprising findings
- Scaling to larger CLIP teachers (ViT-L/14) did not improve results, and ImageNet zero-shot accuracy did not correlate with counting performance.
- Convolutional CLIP teachers (ResNet50, ConvNeXt) performed worse despite higher ImageNet accuracy.
Limitations
- Results depend on the choice of CLIP teacher, and larger teachers did not help.
- Requires category names to generate LLM prompts; the LVLM-prompt variant is less accurate (MAE about 22.5).
Experimental setup
Datasets: FSC-147 FSCD-LVIS
Benchmarks: FSC-147 FSCD-LVIS
Metrics: MAE RMSE
Baselines: FamNet BMNet+ CounTR LOCA DAVE ISW GS DCCUS MPCount SAFECount PseCo VCN
Builds on: CLIP MaskCLIP CuPL CLIP-KD LOCA
Closest papers by meaning
- CountGD: Multi-Modal Open-World Counting (NeurIPS 2024 · similarity 0.88)
- T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting (CVPR 2025 · similarity 0.88)
- VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting (AAAI 2024 · similarity 0.87)
- Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting (AAAI 2024 · similarity 0.87)
- A Novel Unified Architecture for Low-Shot Counting by Detection and Segmentation (NeurIPS 2024 · similarity 0.87)
- DAVE - A Detect-and-Verify Paradigm for Low-Shot Counting (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.