Research map / Point clouds / Point cloud classification
Point Transformer V3: Simpler Faster Stronger
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Point Transformer V3, a simplified and scaled point cloud transformer that replaces KNN and relative positional encoding with serialized point ordering, patch attention, and a sparse-conv positional encoding, achieving SOTA on 20+ 3D tasks with better speed and memory.
Problem
Point transformers trade accuracy for efficiency and are slow and memory-hungry (KNN queries and relative positional encoding dominate runtime), which limits receptive field scaling and benefit from large-scale multi-dataset training in 3D perception.
Approach
Prioritize simplicity and efficiency over precise neighbor search: serialize point clouds along space-filling curves (Z-order, Hilbert and transposed variants), apply non-overlapping patch attention with shuffled serialization orders across layers, and replace RPE with a prepended sparse convolution (xCPE), so that scale compensates for lost accuracy.
Key results
- PTv3 reaches 78.6% mIoU on ScanNet validation and 79.4% test with multi-dataset training, outperforming PTv2 by 3.7% on ScanNet test from scratch (77.9 vs 74.2).
- PTv3 is 3.3x faster in inference (44ms vs 146ms for PTv2) and uses 10.2x less memory (1.2G vs 12.3G) on nuScenes, while expanding receptive field from 16 to 1024 points.
- On nuScenes validation PTv3 reaches 81.2% mIoU with pre-training versus 78.4% for SphereFormer trained from scratch (80.4% from scratch).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use all four serialization patterns instead of Z-order only (ablation) | PTv3 with Shift Order · ScanNet semantic segmentation val | mIoU: 74.3 -> 76.5 | Helped |
| replace Shift Order with Shift Dilation (ablation) | PTv3 · ScanNet semantic segmentation val | mIoU: 76.5 -> 76.8 | Helped |
| replace Shift Order with Shift Patch (ablation) | PTv3 · ScanNet semantic segmentation val | mIoU: 76.5 -> 76.6 | Helped |
| use Shuffle Order with all four serialization patterns (ablation) | PTv3 · ScanNet semantic segmentation val | mIoU: 76.5 -> 77.3 | Helped |
| replace CPE with xCPE (ablation) | PTv3 · ScanNet semantic segmentation val | mIoU: 76.8 -> 77.3 | Helped |
| replace RPE with xCPE (ablation) | PTv3 · ScanNet semantic segmentation val | mIoU: 75.9 -> 77.3 | Helped |
| increase patch size from 16 to 1024 (ablation) | PTv3 · ScanNet semantic segmentation val | mIoU: 75.0 -> 77.3 | Helped |
| increase patch size from 1024 to 4096 (ablation) | PTv3 · ScanNet semantic segmentation val | mIoU: 77.3 -> 77.1 | Hurt |
| replace PTv2 with PTv3 | PointGroup on ScanNet · ScanNet instance segmentation | mAP: 38.3 -> 40.9 | Helped |
| replace PTv2 with PTv3 | SemanticKITTI semantic segmentation · SemanticKITTI val | mIoU: 70.3 -> 72.3 | Helped |
Surprising findings
- Dropping precise KNN neighbor search and relative positional encoding in favor of serialized patches yields better accuracy when scaled, despite lower accuracy per mechanism.
- Increasing the attention patch size from 16 to 1024 points keeps latency nearly constant on nuScenes (119ms vs 120ms for 256).
Limitations
- Serialization trades away precise locality-preserving neighbor relations, which lowers accuracy when not scaled.
- Shift Patch and Shift Dilation depend on attention masks that reduce efficiency.
- Gains from multi-dataset joint training rely on PPT-style pretraining that is not detailed in the text.
Experimental setup
Datasets: ScanNet ScanNet200 S3DIS nuScenes SemanticKITTI Waymo PPT multi-dataset joint training
Benchmarks: ScanNet ScanNet200 S3DIS nuScenes SemanticKITTI Waymo PointGroup instance segmentation ScanNet data efficient
Metrics: mIoU mAP mAP25 mAP50 mAPH latency memory
Baselines: PTv2 OctFormer MinkUNet SphereFormer FlatFormer Swin3D Stratified Transformer PointGroup
Builds on: Point Transformer V2 OctFormer FlatFormer Point Prompt Training Sparse Convolution
Closest papers by meaning
- LitePT: Lighter Yet Stronger Point Transformer (CVPR 2026 · similarity 0.91)
- FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer (CVPR 2023 · similarity 0.91)
- Point Transformer V2: Grouped Vector Attention and Partition-based Pooling (NeurIPS 2022 · similarity 0.90)
- Flash3D: Super-scaling Point Transformers through Joint Hardware-Geometry Locality (CVPR 2025 · similarity 0.88)
- DAPoinTr: Domain Adaptive Point Transformer for Point Cloud Completion (AAAI 2025 · similarity 0.87)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.