Research map / Label-efficient and robust learning / Recognition and long-tail
Learning Group Activity Features Through Person Attribute Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns compact group activity features (GAF) without group activity labels by predicting per-person attributes (action classes or appearance features) from masked person features with location-guided attribute prediction, using transformers on VBD and CAD.
Problem
Group activity recognition and retrieval normally require manually annotated group activity labels, which are costly and hard to define for fine-grained activity classes.
Approach
Learn a group activity feature by requiring it to predict each person's attributes (action class or appearance features) from location-guided queries, with masked person modeling and dual spatial-temporal transformer branches, so no group activity labels are needed.
Key results
- On VBD in GAFL-PAC, Ours-grp reaches 84.8% Hit@1 group activity retrieval vs. 64.4% for Dual-AI.
- On CAD in GAFL-PAF, Ours-ind reaches 88.5% Hit@1 group activity retrieval vs. 72.2% for B2-VGG19.
- Fine-tuning the GAF-pretrained network for group activity recognition gives 96.6% vs. 94.1% for Dual-AI on CAD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn GAF through person action prediction without group activity labels | GAFL-PAC · VBD group activity retrieval | Hit@1: 57.0 -> 84.8 | Helped |
| Learn GAF through person appearance-feature prediction | GAFL-PAF · VBD group activity retrieval | Hit@1: 35.4 -> 61.1 | Helped |
| Learn GAF through person action prediction without group activity labels | GAFL-PAC · CAD group activity retrieval | Hit@1: 90.8 -> 94.9 | Helped |
| Learn GAF through person appearance-feature prediction | GAFL-PAF · CAD group activity retrieval | Hit@1: 82.4 -> 88.5 | Helped |
| Add person location guidance to attribute prediction (ablation) | GAFL-PAC · VBD group activity retrieval | Hit@1: 69.5 -> 84.8 | Helped |
| Add person location guidance to attribute prediction (ablation) | GAFL-PAF · VBD group activity retrieval | Hit@1: 53.2 -> 61.1 | Helped |
| Fine-tune the pretrained GAF network with group activity labels | GAF learning network · VBD GAR | accuracy: 92.1 -> 92.4 | Helped |
| Fine-tune the pretrained GAF network with group activity labels | GAF learning network · CAD GAR | accuracy: 94.1 -> 96.6 | Helped |
| Use 5 masked people rather than 0 in person masking (ablation) | GAFL-PAC · VBD GAR by nearest-neighbor retrieval | Hit@1 (group activity): 83.8 -> 85.0 | Helped |
| Use 6 masked people rather than 0 in person masking (ablation) | GAFL-PAF · VBD GAR by nearest-neighbor retrieval | Hit@1 (group activity): 68.1 -> 70.9 | Helped |
| Use 11 masked people (ablation) | GAFL-PAC · VBD GAR by nearest-neighbor retrieval | Hit@1 (group activity): 83.8 -> 78.5 | Hurt |
| Use 11 masked people (ablation) | GAFL-PAF · VBD GAR by nearest-neighbor retrieval | Hit@1 (group activity): 68.1 -> 55.5 | Hurt |
Surprising findings
- Ours-grp underperforms Ours-ind on CAD retrieval, attributed to the number of people per scene affecting feature learning difficulty.
- Large masking ratios (10 or 11 of 12 people masked) degrade performance substantially, indicating that attribute prediction from few unmasked people fails.
Limitations
- Performance remains inferior to group-activity-supervised GAR methods.
- Ours-grp versus Ours-ind trade-off depends on the number of people per scene.
- Evaluated only on two datasets (Volleyball and Collective Activity).
Experimental setup
Datasets: Volleyball dataset (VBD) Collective Activity Dataset (CAD)
Benchmarks: Volleyball dataset Collective Activity Dataset
Metrics: Hit@1 Hit@2 Hit@3 mAP accuracy
Baselines: HiGCIN DIN Dual-AI B1-Compact128 B2-VGG19 HRN
Builds on: Dual-AI DIN HRN Transformer VGG
Closest papers by meaning
- Detector-Free Weakly Supervised Group Activity Recognition (CVPR 2022 · similarity 0.84)
- A Hierarchical Deep Temporal Model for Group Activity Recognition (CVPR 2016 · similarity 0.84)
- Convolutional Relational Machine for Group Activity Recognition (CVPR 2019 · similarity 0.83)
- Learning from Synthetic Human Group Activities (CVPR 2024 · similarity 0.81)
- Learning Group Actions on Latent Representations (NeurIPS 2024 · similarity 0.81)
- Bi-Causal: Group Activity Recognition via Bidirectional Causality (CVPR 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.