Research map / Video / Action recognition
GAIA: Rethinking Action Quality Assessment for AI-Generated Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds GAIA, a 9,180-video dataset of AI-generated human actions from 18 text-to-video models with 971k human ratings along three causal-reasoning (syllogism) dimensions, and benchmarks AQA, action-related T2V metrics, VQA, and video-text alignment metrics against it.
Problem
Existing action quality assessment (AQA) methods and T2V evaluation metrics are built for real, domain-specific videos with expert scalar scores, and they are unclear whether they assess the action quality of AI-generated videos (AIGVs) in line with human perception.
Approach
Reframe AIGV action quality as a causal-reasoning "action syllogism" with three ordered dimensions (subject quality as major premise, action completeness as minor premise, action-scene interaction as conclusion), collect large-scale crowd ratings for each, and use the resulting benchmark to test existing AQA, action metrics, VQA, and alignment metrics.
Key results
- Conventional AQA methods reach average SRCC of about 0.454 (best TSA at 0.4861 all-combined), action-related T2V metrics only about 0.191 (best Human Action 0.2831), and VQA methods about 0.519 (best DOVER 0.5335 all-combined).
- Video-text alignment metrics improve when CLIP is replaced by a VLM: InternLMScore reaches 0.4314 all-combined SRCC, versus 0.3777 for CLIPScore ViT-B/16.
- Commercial T2V models generally outperform lab-study models, and the newly proposed Mora significantly underperforms in all three dimensions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use conventional AQA methods (USDL, ACTION-NET, CoRe, TSA) for action-quality assessment | GAIA videos · GAIA | SRCC: 0.4367, 0.4722, and 0.4664 | Hurt |
| Use action-related metrics from existing T2V benchmarks | GAIA videos · GAIA | SRCC: 0.2453, 0.2895, and 0.2861 | Hurt |
| Use video quality assessment methods for action-quality assessment | GAIA videos · GAIA | SRCC: 25.04% and 131.1% | Helped |
| Replace CLIP with InternLM-XComposer2-VL in video-text alignment scoring | Video-text alignment metrics on GAIA · GAIA | SRCC: +8.2%, +9.6%, +10.9%, +13.1% | Helped |
| Generate higher-resolution videos | Text-to-video models evaluated on GAIA · GAIA | subject quality and action completeness | Helped |
| Increase frame rate | Text-to-video models evaluated on GAIA · GAIA | action quality | Helped |
| Generate complex, high-motion actions rather than small-movement actions | Text-to-video models evaluated on GAIA · GAIA action categories | MOS: p<0.01 | Hurt |
| Generate subtle local hand actions, such as finger/palm movement or numeral representation | Text-to-video models evaluated on GAIA · GAIA hand-action categories | MOS | Hurt |
Surprising findings
- Mora, a recently proposed multi-agent video generation framework, significantly underperforms other models on all three action dimensions, which the authors attribute to its reliance on an earlier image-to-video backbone.
- Dedicated action-related metrics from VBench and EvalCrafter correlate very weakly with human judgments (best SRCC about 0.28), below general VQA methods.
Limitations
- Videos cover limited subject types and styles, and the synthetic actions are simpler than real-life motions.
- Generated videos have limited resolutions, frame rates, and lengths, varying across the commercial and lab models.
- Annotations use a novel causal-reasoning syllogism scheme that departs from conventional single-score AQA labels, so its usefulness for AQA is untested.
Experimental setup
Datasets: Kinetics-400 EgoGesture GAIA
Benchmarks: GAIA
Metrics: SRCC PLCC
Baselines: USDL ACTION-NET CoRe TSA VBench action metrics EvalCrafter Action-Score TLVQM VIDEVAL VSFA BVQA SimpleVQA FAST-VQA
Builds on: VBench EvalCrafter FAST-VQA DOVER CoRe
Closest papers by meaning
- FineParser: A Fine-grained Spatio-temporal Action Parser for Human-centric Action Quality Assessment (CVPR 2024 · similarity 0.88)
- AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM (CVPR 2025 · similarity 0.87)
- LucidAction: A Hierarchical and Multi-model Dataset for Comprehensive Action Quality Assessment (NeurIPS 2024 · similarity 0.86)
- Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs (CVPR 2025 · similarity 0.85)
- UVE: Are MLLMs Unified Evaluators for AI-Generated Videos? (NeurIPS 2025 · similarity 0.85)
- VE-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.