Research map / Point clouds / Point cloud classification
PvNeXt: Rethinking Network Design and Temporal Motion for Point Cloud Video Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PvNeXt, a point cloud video recognition network that replaces dense iterative temporal querying with a Motion Imitator and a Single-Step Motion Encoder that synthesizes virtual frames for one-shot per-frame queries.
Problem
Point cloud video recognition methods rely on iterative temporal looping and dense neighbor queries across frames, which incur heavy computation and memory costs.
Approach
Learn anchor-wise cross-frame motion, use it to synthesize virtual frames, and let each frame perform a single query against its virtual frame, replacing frame-by-frame looping and dense querying with one-shot queries.
Key results
- On MSR-Action3D, PvNeXt reaches 94.77% accuracy at 24 frames and 93.93% at 16 frames with 0.72M parameters and 0.55 GFLOPs (16 frames), versus 44.13M parameters for PST-Transformer.
- On NTU-RGBD cross-subject, PvNeXt alone achieves 89.2%, and adding its branch to PST-Transformer raises accuracy from 91.0% to 91.4%.
- Inference speed on MSR-Action3D (16-frame) is reported as 1735.45 samples/s versus 75.33 for PST-Transformer, about 23x faster; memory is reduced 1.67x relative to PST-Transformer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PvNeXt instead of PST-Transformer | MSR-Action3D, 16-frame · MSR-Action3D | accuracy (%): 91.98 -> 93.93 | Helped |
| Replace PST-Transformer with PvNeXt | MSR-Action3D, 16-frame · MSR-Action3D | inference speed (samples/s): 75.33 -> 1735.45 | Helped |
| Use PvNeXt instead of PST-Transformer | NTU-RGBD cross-subject · NTU-RGBD | accuracy (%): 91.0 -> 89.2 | Hurt |
| Add the PvNeXt one-step query branch | PST-Transformer · NTU-RGBD | accuracy (%): 91.0 -> 91.4 | Helped |
| Use PvNeXt instead of PPTr | HOI4D action segmentation, 150-frame · HOI4D | accuracy: 77.4 -> 78.5 | Helped |
| Remove the Motion Imitator while retaining feature encoding (ablation) | PvNeXt · MSR-Action3D, 16-frame | accuracy (%): 93.93% -> 86.86% | Hurt |
| Reverse the movement trajectories (ablation) | PvNeXt · MSR-Action3D, 16-frame | accuracy (%): 93.93% -> 92.25% | Hurt |
| Increase Motion Imitator neighbors from 3 to 7 (ablation) | PvNeXt · MSR-Action3D, 16-frame | accuracy (%): 1.00% lower; 92.93% with 7 neighbors | Hurt |
| Add dense querying for 4-frame input (ablation) | PvNeXt · MSR-Action3D, 4-frame | accuracy | Helped |
| Use the learned Motion Imitator rather than nearest-neighbor sampling | PvNeXt Motion Imitator · MSR-Action3D motion prediction | Chamfer Distance: 5.23e-3 -> 1.35e-3 | Helped |
| Use PvNeXt instead of PST-Transformer under occlusion | MSR-Action3D occlusion setting · MSR-Action3D, occlusion | accuracy_occ: 86.53 -> 88.22 | Helped |
Surprising findings
- Reversing the learned motion trajectories drops accuracy from 93.93% to 92.25%, suggesting the learned motion direction matters.
- The method underperforms on very short 4-frame sequences (needs a dense-query variant to reach 80.47%) and offers limited gains on action segmentation.
Limitations
- One-shot querying may lose geometric detail compared with dense querying methods.
- Weaker on very short sequences (4 frames) and on dense action segmentation tasks.
- NTU-RGBD result (89.2%) trails the strongest point-based baseline PSTNet++ (91.4%) when used alone.
Experimental setup
Datasets: MSR-Action3D NTU-RGBD HOI4D
Benchmarks: MSR-Action3D NTU-RGBD HOI4D
Metrics: accuracy FLOPs parameters memory training speed inference speed chamfer distance edit score F1@10 F1@25
Baselines: MeteorNet PSTNet P4Transformer PSTNet++ PST-Transformer Kinet PPTr LeaF 3DInAction MAMBA4D PointCMP PointCPSC
Builds on: PSTNet PointNet++ PST-Transformer PointNeXt
Closest papers by meaning
- Adapting Pre-trained 3D Models for Point Cloud Video Understanding via Cross-frame Spatio-temporal Perception (CVPR 2025 · similarity 0.88)
- 3DInAction: Understanding Human Actions in 3D Point Clouds (CVPR 2024 · similarity 0.87)
- X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-Modal Knowledge Transfer (AAAI 2024 · similarity 0.86)
- A Unified Framework for Human-centric Point Cloud Video Understanding (CVPR 2024 · similarity 0.85)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.85)
- Point Transformer V3: Simpler Faster Stronger (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.