Research map / Architectures and efficiency / Attention and transformers

Patch Hierarchical Attention Transformers for Efficient Particle Jet Tagging

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Sparse efficient transformers for long context

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PHAT-JeT, a parameter- and FLOP-constrained jet-tagging transformer combining geometric (eta-phi grid) message passing, exact intra-patch attention, and patch-token global attention, reporting SOTA among trigger-scale models on four jet tagging benchmarks.

Problem

Real-time LHC trigger jet tagging needs accurate transformers under strict latency, FLOP and parameter budgets, but full transformers are quadratic in particle count and existing efficient variants lose classification accuracy.

Approach

Combine a depthwise-conv geometric message-passing module on a coarse (eta,phi) grid with block-wise exact attention over fixed-size particle patches, plus a global attention stage over mean-pooled patch tokens broadcast back to particles.

Key results

• On hls4ml, PHAT-JeT reaches 81.80% accuracy and 71.6 average background rejection with 6.7K params and 1.31M FLOPs, beating JEDI-Linear (81.56%, 68.7).
• On JetClass, PHAT-JeT achieves 65.38% accuracy and 43.94 background rejection versus 59.52% and 21.93 for JEDI-Linear at ~1.24M FLOPs.
• On Top Tagging, PHAT-JeT reaches 92.69% accuracy and 34.3 background rejection, versus 92.52% and 31.8 for SAL-T.

Surprising findings

• Patch-based attention without GMP slightly outperforms the full O(N^2) Transformer (81.51% vs 81.27%) at roughly half the FLOPs.
• Train/test ordering mismatch degrades performance sharply, so PHAT-JeT is not deployment-time permutation invariant despite being robust to the choice of consistent ordering.

Limitations

• Evaluated only in software; no FPGA/hls4ml hardware synthesis or latency measurements reported.
• Performance is sensitive to train/test ordering mismatch, so it is not permutation invariant at deployment.
• Resource-constrained comparisons use a single attention layer and matched FLOPs, limiting generality to larger budgets.

Experimental setup

Datasets: hls4ml LHC Jet Dataset JetClass Top Tagging Quark-Gluon

Benchmarks: hls4ml LHC Jet Dataset JetClass Top Tagging Quark-Gluon

Metrics: accuracy ROC AUC background rejection at 80% signal efficiency parameters FLOPs

Baselines: JEDI-Linear Linformer Transformer SAL-T Point Transformer V3 HEPT PointNet ParT LorentzNet L-GATr

Builds on: Particle Transformer Point Transformer V3 JEDI-linear Linformer Swin

Closest papers by meaning

• Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE (NeurIPS 2026 · similarity 0.80)
• Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models (ICLR 2024 · similarity 0.80)
• FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning (ICLR 2024 · similarity 0.80)
• Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling (ICML 2026 · similarity 0.80)
• ToMA: Token Merge with Attention for Diffusion Models (ICML 2025 · similarity 0.79)
• Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search (NeurIPS 2025 · similarity 0.79)