Research map / Video / Action recognition
FineSports: A Multi-person Hierarchical Sports Video Dataset for Fine-grained Action Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds FineSports, a 10,000-video NBA basketball dataset with two-level fine-grained action and spatial-temporal annotations, and proposes PoSTAL, a prompt-driven spatial-temporal action localization model that outperforms MOC and TubeR on it.
Problem
Fine-grained action understanding and spatial-temporal action localization in multi-person sports are hard due to occlusion and rapid movement, and existing sports datasets lack fine-grained spatial-temporal annotations.
Approach
Formulates STAL as prompt-driven localization: a target player is described by jersey color and number, text embeddings (BERT) are cross-attended with video features in a prompt-driven target action encoder, and an action tube-specific detector regresses N tubes and their fine-grained classes directly without offline proposals.
Key results
- PoSTAL reaches 21.54 frame-mAP@0.5, versus 19.48 for TubeR and 19.21 for MOC on FineSports.
- PoSTAL reaches 24.31 video-mAP@0.5, versus 17.76 for TubeR.
- Removing descriptive words drops video-mAP@0.5 from 24.31 to 12.44 in the PTA ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PoSTAL instead of TubeR | FineSports spatial-temporal action localization · FineSports | frame-mAP@0.5: 19.48 -> 21.54 | Helped |
| use PoSTAL instead of TubeR | FineSports spatial-temporal action localization · FineSports | video-mAP@0.2: 28.91 -> 31.18 | Helped |
| use PoSTAL instead of TubeR | FineSports spatial-temporal action localization · FineSports | video-mAP@0.5: 17.76 -> 24.31 | Helped |
| use PoSTAL instead of MOC | FineSports spatial-temporal action localization · FineSports | frame-mAP@0.5: 19.21 -> 21.54 | Helped |
| add target-player descriptive words (ablation) | PoSTAL · FineSports | frame-mAP@0.5: 20.67 -> 21.54 | Helped |
| add target-player descriptive words (ablation) | PoSTAL · FineSports | video-mAP@0.2: 24.69 -> 31.18 | Helped |
| add target-player descriptive words (ablation) | PoSTAL · FineSports | video-mAP@0.5: 12.44 -> 24.31 | Helped |
| use learnable word embeddings (ablation) | PoSTAL · FineSports | frame-mAP@0.5: 19.45 -> 21.54 | Helped |
| use learnable word embeddings (ablation) | PoSTAL · FineSports | video-mAP@0.2: 31.37 -> 31.18 | No clear effect |
| use learnable word embeddings (ablation) | PoSTAL · FineSports | video-mAP@0.5: 18.19 -> 24.31 | Helped |
| use 6 tube queries instead of 2 (ablation) | PoSTAL · FineSports | frame-mAP@0.5: 20.16 -> 21.54 | Helped |
| use 6 tube queries instead of 2 (ablation) | PoSTAL · FineSports | video-mAP@0.2: 32.72 -> 31.18 | Hurt |
Surprising findings
- Without descriptive words, frame-mAP@0.5 drops only 0.87 points (20.67 vs 21.54), while video-mAP@0.5 drops by about half.
Limitations
- Dataset covers only NBA basketball games and does not generalize to other multi-person sports.
- Overall localization accuracy is low (about 21% frame-mAP), reflecting the difficulty of the benchmark.
Experimental setup
Datasets: FineSports Kinetics-400
Benchmarks: FineSports
Metrics: frame-mAP@0.5 video-mAP@0.2 video-mAP@0.5
Baselines: MOC TubeR
Builds on: TubeR MOC CSN BERT BLIP
Closest papers by meaning
- BASKET: A Large-Scale Video Dataset for Fine-Grained Skill Estimation (CVPR 2025 · similarity 0.89)
- FineParser: A Fine-grained Spatio-temporal Action Parser for Human-centric Action Quality Assessment (CVPR 2024 · similarity 0.87)
- ActionAtlas: A VideoQA Benchmark for Domain-specialized Action Recognition (NeurIPS 2024 · similarity 0.86)
- ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysis (AAAI 2024 · similarity 0.86)
- SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videos (CVPR 2024 · similarity 0.86)
- Towards Universal Soccer Video Understanding (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.