Research map / 3D vision / Human pose estimation
3DInAction: Understanding Human Actions in 3D Point Clouds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3DinAction, a point-cloud action recognition pipeline built on temporal patches (t-patches) that track local point neighborhoods over time with a hierarchical MLP/conv network, evaluated per-frame on DFAUST, IKEA ASM and MSR-Action3D.
Problem
Human action recognition from 3D point cloud sequences is under-explored because point clouds are unstructured, unordered, have varying point counts, and lack point-level temporal correspondence.
Approach
Instead of relying on ground-truth correspondence, grow local patches over time by iteratively taking kNN/nearest neighbors in each next frame (t-patches), add jitter and bidirectional extraction to mitigate patch collapse, and learn a hierarchical spatio-temporal network over them.
Key results
- On DFAUST, 3DinAction reaches 87.26% frame-wise top-1 accuracy versus 74.10% for PointNet+TS, a large margin over all baselines.
- On IKEA ASM, the method reaches 52.91% top-1 accuracy and 0.2875 mAP, vs 45.97% top-1 for PointNet++.
- On MSR-Action3D, results are comparable to SoTA (e.g., 80.47 to 92.23 across 4 to 24 frames), which the authors attribute to dataset scale.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use t-patches with the hierarchical 3DinAction architecture rather than the strongest listed baseline | DFAUST · DFAUST | top-1 frame accuracy: 74.10 -> 87.26 | Helped |
| use t-patches with the hierarchical 3DinAction architecture rather than PointNet++ | IKEA ASM · IKEA ASM | top-1 frame accuracy: 45.97 -> 52.91 | Helped |
| use 3DinAction rather than PST-Transformer | MSR-Action3D (8 frames) · MSR-Action3D (8 frames) | classification accuracy: 83.97 -> 86.20 | Helped |
| use 3DinAction rather than PST-Transformer | MSR-Action3D (16 frames) · MSR-Action3D (16 frames) | classification accuracy: 91.98 -> 90.57 | Hurt |
| add jitter during t-patch construction (ablation) | 3DinAction on clean DFAUST · DFAUST | top-1 frame accuracy: 74.73 -> 80.49 | Helped |
| add bidirectional t-patches (ablation) | 3DinAction with jitter on clean DFAUST · DFAUST | top-1 frame accuracy: 80.49 -> 87.26 | Helped |
| use ground-truth correspondence instead of nearest-neighbor matching (ablation) | 3DinAction on clean DFAUST · DFAUST | top-1 frame accuracy: 74.73 -> 77.67 | Helped |
| add jitter during t-patch construction (ablation) | 3DinAction on noisy DFAUST · DFAUST noisy | top-1 frame accuracy: 66.74 -> 81.83 | Helped |
| add bidirectional t-patches (ablation) | 3DinAction with jitter on noisy DFAUST · DFAUST noisy | top-1 frame accuracy: 81.83 -> 80.03 | Hurt |
| increase sampled points from n=512 to n=1024 (k=16) (ablation) | 3DinAction on noisy DFAUST · DFAUST noisy | top-1 frame accuracy: 80.03 -> 77.30 | Hurt |
| increase neighbors per patch from k=16 to k=32 (n=512) (ablation) | 3DinAction on noisy DFAUST · DFAUST noisy | top-1 frame accuracy: 80.03 -> 77.91 | No clear effect |
Surprising findings
- Using ground-truth point correspondence (Ours + GT corr) gives lower accuracy than the simplified kNN-based t-patch extraction on DFAUST and in the noisy ablation.
- Top-1 accuracy is not indicative of quality on IKEA ASM because of class imbalance; a high top-1 corresponds to predicting the most common class.
Limitations
- kNN-based t-patches are sensitive to variations in point density.
- Very low frame rates or fast motion break the assumption that consecutive frame points are close, causing inconsistent t-patch motion.
- Extraction of t-patches (FPS and sequential kNN) dominates runtime.
Experimental setup
Datasets: DFAUST IKEA ASM MSR-Action3D
Benchmarks: DFAUST IKEA ASM MSR-Action3D
Metrics: frame-wise top-1 accuracy top-3 accuracy macro recall mAP
Baselines: PointNet PointNet++ Set Transformer PSTNet 3DmFVNet PointNet+TS PointNet+++TS Set Transformer+TS P4Transformer PST-Transformer Kinet HCN
Builds on: PointNet PointNet++ PSTNet GradCAM Set Transformer
Closest papers by meaning
- Adapting Pre-trained 3D Models for Point Cloud Video Understanding via Cross-frame Spatio-temporal Perception (CVPR 2025 · similarity 0.90)
- A Unified Framework for Human-centric Point Cloud Video Understanding (CVPR 2024 · similarity 0.88)
- PvNeXt: Rethinking Network Design and Temporal Motion for Point Cloud Video Recognition (ICLR 2025 · similarity 0.87)
- X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-Modal Knowledge Transfer (AAAI 2024 · similarity 0.86)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.86)
- Point-Cache: Test-time Dynamic and Hierarchical Cache for Robust and Generalizable Point Cloud Analysis (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.