Research map / Video / Action recognition
3D-aware Select, Expand, and Squeeze Token for Aerial Action Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3D-Tok, a transformer framework that selects top-K tokens along temporal, height, and width dimensions (3TS) and expands/squeezes them via 3D conv interpolation with contrastive losses (ESC) for aerial action recognition.
Problem
Aerial action recognition in UAV videos, where humans occupy small regions and UAV motion disturbs relative movement, yielding sparse and disturbed semantic information that standard action recognition models handle poorly.
Approach
Select compact yet diverse tokens jointly along the T, H, and W dimensions of a 3D token cube via learned semantic importance scores with Top-K, then expand and squeeze the selected tokens with 3D convolution/deconvolution interpolation and attention, aligned by supervised contrastive losses.
Key results
- On UAV-Human (16 frames, 224x224), 3D-Tok reaches 64.5% accuracy vs. 55.0% for PMI Sampler (+9.5).
- On Drone-Action, 3D-Tok reaches 98.7% accuracy, a +5.6 gain over ASAT (93.1%).
- On RoCoG-v2, 3D-Tok reaches 48.6% (8 frames) and 63.7% (16 frames), improving over AZTR by 19.1 and 25.3 points respectively under the paper's frame settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3D-Tok instead of PMI Sampler | Aerial action recognition · UAV-Human, 16 frames, 224 × 224 | accuracy: 55.0 -> 64.5 | Helped |
| Use 3D-Tok instead of ASAT | Aerial action recognition · Drone-Action | accuracy: 93.1 -> 98.7 | Helped |
| Use 3D-Tok instead of AZTR | Aerial action recognition · RoCoG-v2, 8 frames | accuracy: 29.5 -> 48.6 | Helped |
| Add 3TS (ablation) | 3D-Tok on UAV-Human · UAV-Human | accuracy: 58.8 -> 60.2 | Helped |
| Add ESC (ablation) | 3D-Tok on UAV-Human · UAV-Human | accuracy: 58.8 -> 63.9 | Helped |
| Add 3TS and ESC (ablation) | 3D-Tok on Drone-Action · Drone-Action | accuracy: 90.5 -> 98.7 | Helped |
| Add both contrastive losses Le and Ls (ablation) | 3D-Tok on UAV-Human · UAV-Human | accuracy: 63.7 -> 64.5 | Helped |
| Add both contrastive losses Le and Ls (ablation) | 3D-Tok on Drone-Action · Drone-Action | accuracy: 96.5 -> 98.7 | Helped |
| Set token selection ratio r to 0.5 instead of 0.8 (ablation) | 3D-Tok · UAV-Human | accuracy: 64.5 -> 62.7 | Hurt |
| Set token selection ratio r to 0.9 instead of 0.8 (ablation) | 3D-Tok · UAV-Human | accuracy: 64.5 -> 64.3 | Hurt |
| Set token selection ratio r to 1 instead of 0.8 (ablation) | 3D-Tok · UAV-Human | accuracy: 64.5 -> 64.3 | Hurt |
Surprising findings
- Using more tokens (ratio r=0.9) or all tokens (r=1) decreases performance, which the authors attribute to cluttered information.
Limitations
- Expansion/squeezing is only implemented along the temporal dimension in experiments; spatial/two- or three-dimensional variants are not evaluated.
- Authors state the technical designs may not be optimal and the method is performance-driven rather than efficiency-driven.
- RoCoG-v2 comparison uses different frame counts and input sizes across baselines, complicating direct comparison.
Experimental setup
Datasets: UAV-Human Drone-Action RoCoG-v2
Benchmarks: UAV-Human Drone-Action RoCoG-v2
Metrics: accuracy
Baselines: I3D-M X3D-M FAR DiffFAR PMI Sampler MITFAS MG Sampler TimeSformer ASAT AZTR MoViNet A3 SlowFast
Builds on: Transformer supervised contrastive loss TimeSformer MG Sampler PMI Sampler
Closest papers by meaning
- Multiview Aerial Visual RECognition (MAVREC): Can Multi-view Improve Aerial Visual Perception? (CVPR 2024 · similarity 0.84)
- UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles (NeurIPS 2024 · similarity 0.84)
- Dynamic Semantic-Aware Correlation Modeling for UAV Tracking (NeurIPS 2025 · similarity 0.84)
- Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology (ICLR 2025 · similarity 0.83)
- A2Seek: Towards Reasoning-Centric Benchmark for Aerial Anomaly Understanding (NeurIPS 2025 · similarity 0.82)
- LVPTrack: High Performance Domain Adaptive UAV Tracking with Label Aligned Visual Prompt Tuning (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.