Research map / Video / Video-language understanding
BASKET: A Large-Scale Video Dataset for Fine-Grained Skill Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BASKET, a 4,477-hour basketball video dataset of 32,232 players labeled on 20 fine-grained skills (5 levels), and benchmarks video recognition models, finding best accuracy of 28.50% (VideoMamba) versus 20% random and about 72% for top human experts.
Problem
Fine-grained skill estimation from long videos requires large-scale, diverse data, which existing skill assessment datasets lack; current video models struggle to rate player skill levels.
Approach
A very large, demographically and geographically diverse basketball highlight-video dataset where each player's ~9-minute highlight video is labeled with expert-derived ranks for 20 skills, framed as multi-way classification over 5 skill levels per skill.
Key results
- VideoMamba achieves the best test accuracy of 28.50% on BASKET (64 frames), versus a 20% random baseline; all evaluated models score below 30%.
- In a human study on a 3-level simplified task, VideoMamba reached 41% while novices averaged 43%, and experts averaged 66% (top 72%).
- Cross-league generalization drops VideoMamba accuracy to 23.09% on unseen leagues, versus 28.50% in-domain.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 64 input frames instead of 16 or 32 (ablation) | VideoMamba · BASKET test | top-1 accuracy (%): 23.93 -> 28.50; 25.20 -> 28.50 | Helped |
| Use 128 input frames instead of 64 (ablation) | VideoMamba · BASKET test | top-1 accuracy (%): 28.50 -> 27.18 | Hurt |
| Use uniform frame sampling instead of sampling rates 32, 64, or 128 (ablation) | VideoMamba · BASKET test | top-1 accuracy (%): 22.64 -> 28.50; 25.89 -> 28.50; 26.30 -> 28.50 | Helped |
| Train with the full dataset instead of a 25%, 50%, or 75% subset (ablation) | VideoMamba · BASKET test | top-1 accuracy (%): 23.67 -> 28.50; 24.89 -> 28.50; 27.76 -> 28.50 | Helped |
| Resample the 50 clips used to create player highlight videos (ablation) | VideoMamba · BASKET test | top-1 accuracy (%): 28.44; variance of 1.30; reported accuracy 28.50% | No clear effect |
| Evaluate on previously unseen leagues | VideoMamba · BASKET cross-league generalization | top-1 accuracy (%): 28.50 -> 23.09 | Hurt |
| Evaluate on a previously unseen season and league | VideoMamba · BASKET cross-season and cross-league generalization | top-1 accuracy (%): 28.50 -> 23.53 | Hurt |
Surprising findings
- Vision-language models such as LLaVA-OneVision and SigLIP are among the worst performers on BASKET.
- Randomly resampling highlight clips per player yields nearly identical accuracy (28.44% average vs 28.50%), suggesting clip selection has little effect.
Limitations
- Models perform only slightly above random chance, with poor out-of-domain generalization to unseen leagues.
- Human study uses a simplified 3-level setting on 5 skills and 5 players, not the full 5-level 20-skill task.
- Skill labels are derived from expert event annotations and relative ranking within league and season rather than absolute skill.
Experimental setup
Datasets: BASKET
Benchmarks: BASKET
Metrics: top-1 accuracy averaged across 20 skills
Baselines: TimeSformer MeMViT VideoMAEv2 Unmasked Teacher InternVideo2 VideoMamba X-CLIP SigLIP LLaVA-OneVision
Builds on: Kinetics Ego-Exo4D
Closest papers by meaning
- FineSports: A Multi-person Hierarchical Sports Video Dataset for Fine-grained Action Understanding (CVPR 2024 · similarity 0.89)
- SkillMimic: Learning Basketball Interaction Skills from Demonstrations (CVPR 2025 · similarity 0.85)
- Learning Skill-Attributes for Transferable Assessment in Video (NeurIPS 2025 · similarity 0.85)
- Towards Universal Soccer Video Understanding (CVPR 2025 · similarity 0.83)
- ExpertAF: Expert Actionable Feedback from Video (CVPR 2025 · similarity 0.83)
- LucidAction: A Hierarchical and Multi-model Dataset for Comprehensive Action Quality Assessment (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.