Research map / Graph learning / Graph self-supervised learning
Gramformer: Learning Crowd Counting via Graph-Modulated Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Gramformer, a graph-modulated transformer for crowd counting that diversifies attention via an anti-similarity attention graph and modulates node features with feature-based centrality encoding, reaching state-of-the-art MAE on UCF-QNRF, JHU++ and NWPU.
Problem
Transformers used for crowd counting produce homogenized self-attention maps because crowd images contain many similar patches, which limits counting accuracy.
Approach
Build an attention graph from dissimilarities between patches (via an Edge Weight Regression network with a horizontal-line regularizer) to modulate attention in an anti-similarity fashion, and a feature-based kNN graph whose node in-degree (centrality index) selects learnable embeddings that bias node features in each layer.
Key results
- Gramformer reaches 76.7 MAE and 129.5 MSE on UCF-QNRF, versus 77.3/131.5 for MAN, and 53.1 MAE on JHU++.
- On NWPU-CROWD, Gramformer achieves 72.5 MAE, 316.4 MSE and 0.160 NAE, the best overall in the table.
- Ablation on UCF-QNRF: EWR and centrality indices together reduce MAE from 84.7 baseline to 76.7; edge regularization adds a further gain.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Gramformer instead of the strongest listed baseline | Crowd counting models · ShanghaiTech A test | MAE: 52.7 -> 54.7 | Hurt |
| Use Gramformer instead of the strongest listed baseline | Crowd counting models · UCF-QNRF test | MAE: 76.7 -> 76.7 | No clear effect |
| Use Gramformer instead of the strongest listed baseline | Crowd counting models · JHU-Crowd++ test | MAE: 53.4 -> 53.1 | Helped |
| Use Gramformer instead of the strongest listed baseline | Crowd counting models · NWPU-CROWD test | MAE: 76.5 -> 72.5 | Helped |
| Add EWR to the VGG+IAL baseline (ablation) | Gramformer · UCF-QNRF | MAE: 84.7 -> 79.3 | Helped |
| Add Centrality Indices to the VGG+IAL baseline (ablation) | Gramformer · UCF-QNRF | MAE: 84.7 -> 79.8 | Helped |
| Add Edge Regularization to the model with EWR and Centrality Indices (ablation) | Gramformer · UCF-QNRF | MAE: 77.9 -> 76.7 | Helped |
| Replace graph-guided attention with Graphormer attention (ablation) | Gramformer · UCF-QNRF | MAE: 76.7 -> 80.9 | Hurt |
| Change centrality bank size from 18 to 36 or 12 (ablation) | Gramformer · UCF-QNRF | MAE: 76.7 -> 80.2 | Hurt |
| Change centrality bank size from 18 to 24 (ablation) | Gramformer · UCF-QNRF | MAE: 76.7 -> 78.4 | Hurt |
| Change nearest-neighbor percentage from 0.3 to 0.06, 0.08, 0.1, 0.2, or 0.4 (ablation) | Gramformer · UCF-QNRF | MAE: 76.7 -> 81.6; 78.3; 78.8; 77.9; 78.2 | Hurt |
| Set regularization weight λ to 0.001, 0.01, or 0.1 rather than 0 (ablation) | Gramformer · UCF-QNRF | MAE: 77.9 -> 77.2; 76.8; 76.7 | Helped |
Surprising findings
- Setting the regularization weight lambda to 1 hurts performance, even worse than no regularization.
- Vanilla transformer attention maps on crowd images have ANVar more than 10x lower than on ImageNet, indicating homogenized attention.
Limitations
- Slight increase in FLOPs and inference time from kNN computation in forward pass.
- Performance sensitive to hyperparameters m (bank size) and q (neighbor percentage), requiring tuning.
- On sparse ShanghaiTech A, method ranks second rather than best.
Experimental setup
Datasets: ShanghaiTech A UCF-QNRF NWPU-CROWD JHU-CROWD++
Benchmarks: ShanghaiTech A UCF-QNRF JHU-CROWD++ NWPU-CROWD
Metrics: MAE MSE NAE
Baselines: MCNN CSRNet SANet CAN BL DM-Count UOT GL S3 ChfL MAN CLTR
Builds on: Graphormer MAN Instance Attention Loss
Closest papers by meaning
- Boosting Crowd Counting via Multifaceted Attention (CVPR 2022 · similarity 0.90)
- Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks (CVPR 2019 · similarity 0.89)
- Attention Scaling for Crowd Counting (CVPR 2020 · similarity 0.88)
- Hybrid Graph Neural Networks for Crowd Counting (AAAI 2020 · similarity 0.88)
- Almost Unsupervised Learning for Dense Crowd Counting (AAAI 2019 · similarity 0.88)
- Free Lunch Enhancements for Multi-modal Crowd Counting (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.