Research map / Point clouds / Point cloud classification
Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HEPT, an LSH-based efficient point transformer combining E2LSH with OR-and-AND constructions and point-coordinate hash codes for regular block-sparse attention, evaluated on HEP tracking and pileup tasks.
Problem
Efficient transformers for large-scale point clouds with local inductive bias (e.g., HEP particle tracking and pileup mitigation) suffer from approximation error versus compute tradeoffs and irregular computation of GNNs and prior sparse attention methods.
Approach
Analyzes error-complexity tradeoffs showing RFF and OR-only LSH are inferior for local-inductive-bias tasks, then uses an E2LSH kernel with OR & AND LSH hash codes plus point coordinates as shared AND codes to align query/key buckets, enabling regular block-diagonal attention with near-linear complexity.
Key results
- HEPT achieves AP@k 92.66 on Tracking-6k and 91.93 on Tracking-60k, outperforming SOTA GNNs (91.00 and 90.89) and all transformer baselines.
- On Tracking-60k HEPT gives 89-203x inference speedup and 3-6x training speedup over GNNs, per the paper.
- On Pileup-10k HEPT reaches AUC 40.39, versus 40.26 for the SOTA GNN and 36.70 for the best efficient transformer baseline (Reformer).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HEPT instead of the strongest reported baseline | Tracking-6k · Tracking-6k | AP@k: 91.00 -> 92.66 | Helped |
| Use HEPT instead of the strongest reported baseline | Tracking-60k · Tracking-60k | AP@k: 90.89 -> 91.93 | Helped |
| Use HEPT instead of SMYRF with the HEPT kernel | Pileup-10k · Pileup-10k | AUC: 40.31 -> 40.39 | Helped |
| Use HEPT for test-time inference instead of the fastest baseline transformer | Tracking-6k · Tracking-6k | test time (ms/sample): 8.3 -> 7.0 | Helped |
| Use HEPT for test-time inference instead of FlatFormer | Tracking-60k · Tracking-60k | test time (ms/sample): 58.7 -> 57.9 | Helped |
| Use HEPT for test-time inference instead of FlatFormer | Pileup-10k · Pileup-10k | test time (ms/sample): 12.2 -> 10.7 | Helped |
| Add the HEPT kernel instead of the original transformer kernel and absolute positional encoding (ablation) | HEPT on Tracking-60k · Tracking-60k | AP@k: 72.28 -> 91.93 | Helped |
| Add query-key alignment by tying query and key vectors (ablation) | OR-only LSH on Tracking-60k · Tracking-60k | AP@k: 71.42 -> 78.22 | Helped |
| Add query-key alignment by tying query and key vectors (ablation) | OR & AND LSH on Tracking-60k · Tracking-60k | AP@k: 70.98 -> 88.54 | Helped |
| Replace OR-only LSH with OR & AND LSH while using query-key alignment (ablation) | LSH attention on Tracking-60k · Tracking-60k | AP@k: 78.22 -> 88.54 | Helped |
| Increase block size from 50 to 100 with 3 hash tables (ablation) | HEPT on Tracking-60k · Tracking-60k | AP@k: 87.47 -> 91.93 | Helped |
| Increase block size from 100 to 150 with 3 hash tables (ablation) | HEPT on Tracking-60k · Tracking-60k | AP@k: 91.93 -> 92.22 | Helped |
Surprising findings
- Random Fourier feature based methods (Performer, FLT) perform poorly even when given the proposed kernel.
- Using a single hash table (AND-only LSH) performs poorly in practice, consistent with theory.
Limitations
- Speedup in training on Tracking datasets is limited because loss computation dominates training time.
- Evaluated only on two HEP tasks and a synthetic scaling benchmark; generality beyond HEP is not demonstrated.
- Float16 and FlashAttention acceleration not explored.
Experimental setup
Datasets: TrackML Particle Tracking Dataset Pileup-10k (DELPHES simulation)
Benchmarks: Tracking-6k Tracking-60k Pileup-10k
Metrics: AP@k AUC training time per sample inference time per sample FLOPs GPU memory
Baselines: Reformer SMYRF HyperAttention Performer FLT ScatterBrain Point Transformer FlatFormer GCN DGCNN GravNet GatedGNN
Builds on: Reformer E2LSH Random Fourier Features FLT SMYRF
Closest papers by meaning
- LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing (NeurIPS 2024 · similarity 0.85)
- Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems (ICML 2025 · similarity 0.84)
- SE(3)-bi-equivariant Transformers for Point Cloud Assembly (NeurIPS 2024 · similarity 0.84)
- Flash3D: Super-scaling Point Transformers through Joint Hardware-Geometry Locality (CVPR 2025 · similarity 0.83)
- Point Transformer V3: Simpler Faster Stronger (CVPR 2024 · similarity 0.83)
- Weisfeiler Leman for Euclidean Equivariant Machine Learning (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.