Research map / Video / Action recognition
FineParser: A Fine-grained Spatio-temporal Action Parser for Human-centric Action Quality Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FineParser, a spatial-temporal action parser that learns human-centric foreground action representations via mask prediction, step-transition parsing, and fine-grained contrastive regression for action quality assessment, and releases FineDiving-HM mask annotations.
Problem
Action quality assessment (AQA) methods learn video-level representations and lack fine-grained spatial-temporal understanding, which limits credibility and interpretability for applications like Olympic diving scoring.
Approach
Parse each action video in space (predict human-centric foreground action masks with multi-scale I3D upsampling) and in time (PSNet-based step transitions), combine with a static ResNet encoder, and regress scores via pairwise cross-attention between query and exemplar steps.
Key results
- On FineDiving-HM, FineParser reaches Spearman's rho 0.9435 and R-l2 0.2602, versus CoRe 0.9308 and TSA 0.9324.
- On MTL-AQA, FineParser reaches Spearman's rho 0.9585, above CoRe at 0.9512.
- Temporal parsing AIoU@0.75 improves from 0.5007 (TSA) to 0.9467 for FineParser.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FineParser instead of TSA | AQA · FineDiving-HM | ρ: 0.9324 -> 0.9435 | Helped |
| use FineParser instead of CoRe | AQA · MTL-AQA | ρ: 0.9512 -> 0.9585 | Helped |
| add SAP (ablation) | FineParser · FineDiving-HM | ρ: 0.9351 -> 0.9435 | Helped |
| add TAP (ablation) | FineParser · FineDiving-HM | ρ: 0.9313 -> 0.9435 | Helped |
| add SVE (ablation) | FineParser · FineDiving-HM | ρ: 0.9328 -> 0.9435 | Helped |
| set SVE step duration to 2 instead of 4 (ablation) | FineParser · FineDiving-HM | ρ: 0.9435 -> 0.9320 | Hurt |
| set SVE step duration to 8 instead of 4 (ablation) | FineParser · FineDiving-HM | ρ: 0.9435 -> 0.9337 | Hurt |
| replace SVE backbone with ResNet18 (ablation) | FineParser · FineDiving-HM | ρ: 0.9435 -> 0.9363 | Hurt |
| replace SVE backbone with ResNet50 (ablation) | FineParser · FineDiving-HM | ρ: 0.9435 -> 0.9362 | Hurt |
| replace SVE backbone with ViT-S/16 (ablation) | FineParser · FineDiving-HM | ρ: 0.9435 -> 0.9426 | No clear effect |
Limitations
- Human-centric foreground mask annotations require manual labeling and adjustment.
- Mask annotations are difficult to transfer directly to other competitive sports.
- Evaluation is limited to diving datasets (FineDiving-HM and MTL-AQA).
Experimental setup
Datasets: FineDiving FineDiving-HM MTL-AQA
Benchmarks: FineDiving-HM MTL-AQA
Metrics: Spearman's rank correlation Relative l2 distance AIoU MAE F-measure S-measure
Baselines: C3D-LSTM C3D-AVG MSCADC I3D+MLP USDL MUSDL CoRe TSA-Net
Builds on: I3D PSNet CoRe TSA-Net focal loss
Closest papers by meaning
- GAIA: Rethinking Action Quality Assessment for AI-Generated Videos (NeurIPS 2024 · similarity 0.88)
- FineSports: A Multi-person Hierarchical Sports Video Dataset for Fine-grained Action Understanding (CVPR 2024 · similarity 0.87)
- ActionAtlas: A VideoQA Benchmark for Domain-specialized Action Recognition (NeurIPS 2024 · similarity 0.87)
- LucidAction: A Hierarchical and Multi-model Dataset for Comprehensive Action Quality Assessment (NeurIPS 2024 · similarity 0.86)
- Unsupervised Action Segmentation via Fast Learning of Semantically Consistent Actoms (AAAI 2024 · similarity 0.86)
- Video Action Differencing (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.