Research map / Detection and segmentation / Open-set and few-shot detection
Regressor-Segmenter Mutual Prompt Learning for Crowd Counting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes mPrompt, a mutual prompt learning framework where a head segmenter and a density regressor guide each other (point prompt and context prompt) to reduce crowd-counting bias from point annotation variance.
Problem
Crowd counting regressors trained on density maps from point annotations suffer from annotation variance, causing density map bias and inaccurate spatial context in heavily crowded scenes.
Approach
A shared HRNet backbone feeds a density regressor and a head segmenter that prompt each other: the segmenter is trained with point-derived and box-derived pseudo masks refined by offline/online point prompts, and its masks act as a context prompt loss constraining the regressor's density predictions to fall within predicted head regions; an adapter-based extension applies this to SAM.
Key results
- mPrompt (HRNet variant) reports MAE 52.5 on SHA, 5.8 on SHB, 72.2 on QNRF and 50.2 on NWPU(V), versus 59.4/7.8/85.5/65.7 for the regressor-only baseline.
- On NWPU test, the HRNet mPrompt reaches MAE 62.1 versus 63.7 for STEERER, the best prior method listed, and the SAM-extended variant reaches 66.3.
- The SAM-based variant uses about one quarter of the training parameters and one fifth of the FLOPs of the HRNet version while staying marginally behind it in MAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a segmenter trained with pseudo-mask supervision (ablation) | mPrompt regressor · SHA | MAE: 59.4 -> 58.4 | Helped |
| add offline point prompting (ablation) | mPromptrsg · SHA | MAE: 58.4 -> 54.8 | Helped |
| add online point prompting in addition to offline point prompting (ablation) | mPromptp† · SHA | MAE: 54.8 -> 53.9 | Helped |
| add context prompt loss (ablation) | mPromptrsg · SHA | MAE: 58.4 -> 55.3 | Helped |
| combine point prompting and context prompting (ablation) | mPrompt · SHA | MAE: 59.4 -> 52.5 | Helped |
| use a point-only pretrained segmenter to generate offline pseudo masks instead of the NWPU box-pretrained segmenter (ablation) | mPromptrsg · SHA | MAE: 58.4 -> 58.8 | Hurt |
| replace the HRNet backbone with SAM (ablation) | mPrompt‡ · SHA | MAE: 52.5 -> 53.2 | Hurt |
| add offline and online point prompts plus context prompt (ablation) | mPrompt · NWPU(V) | MAE: 65.7 -> 50.2 | Helped |
| train with increasing box-annotation noise (ablation) | mPrompt‡ · NWPU | MAE | No clear effect |
Surprising findings
- Setting the offline mask prompt to empty still outperforms the regressor-only baseline by a large margin, indicating the mutual prompt gain does not rely on pseudo-mask quality alone.
Limitations
- Performance on NWPU test with the SAM-based variant (66.3 MAE) is worse than the HRNet variant (62.1), so the foundation-model extension trades accuracy for efficiency.
- Relies on box annotations from NWPU to build better segmentation pseudo masks; point-only pseudo masks underperform.
Experimental setup
Datasets: ShanghaiTech A ShanghaiTech B UCF-QNRF NWPU-Crowd
Benchmarks: ShanghaiTech A ShanghaiTech B UCF-QNRF NWPU(V) NWPU(T)
Metrics: MAE RMSE IoU
Baselines: GLoss P2PNet DKPNet SASNeT GauNet CLTR DDC PET STEERER
Builds on: HRNet SAM density map regression mutual information maximization
Closest papers by meaning
- Context-Aware Crowd Counting (CVPR 2019 · similarity 0.85)
- Top-Down Feedback for Crowd Counting Convolutional Neural Network (AAAI 2018 · similarity 0.85)
- Taste More, Taste Better: Diverse Data and Strong Model Boost Semi-Supervised Crowd Counting (CVPR 2025 · similarity 0.84)
- Point-to-Region Loss for Semi-Supervised Point-Based Crowd Counting (CVPR 2025 · similarity 0.83)
- Single Domain Generalization for Crowd Counting (CVPR 2024 · similarity 0.83)
- A Fixed-Point Approach to Unified Prompt-Based Counting (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.