Research map / Detection and segmentation / Tracking
Lattice Boltzmann Model for Learning Real-World Pixel Dynamicity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Lattice Boltzmann Model (LBM), a lightweight online predict-update transformer that tracks pixels via collision-streaming dynamics for real-time point and open-world object tracking.
Problem
Real-time online pixel/point tracking for edge and embodied deployment, where existing methods are offline or window-based semi-online and costly in memory, latency and iterations; also robust object tracking under detection failures.
Approach
Treats query pixels as lattice particles whose d-dimensional distributions evolve via learned collision (deformable attention over dynamic spatial neighbors) and streaming (cross-attention over temporal history), refined by an update step with online visual features across a 3-layer predict-update transformer without iterative refinement; for objects, samples and prunes fine-grained pixels within
Key results
- On TAP-Vid DAVIS, LBM (18M params) reaches 65.1 AJ, matching Track-On (49M, 65.0 AJ) while using ~37% of its parameters.
- LBM runs at 14.3 FPS on NVIDIA Jetson Orin NX Super, a 3.9x speed advantage over Track-On; FP16 TensorRT reaches 49 FPS.
- On OVT-B, LBM reports 56.8 TETA, a +9.2 gain over OVTrack+; on BFT it reaches 74.5 OWTA, +2.0 over NetTrack.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LBM instead of Track-On | online point tracking · TAP-Vid DAVIS | AJ: 65.0 -> 65.1 | Helped |
| Use LBM instead of Track-On | online point tracking · TAP-Vid Kinetics | AJ: 53.9 -> 53.4 | Hurt |
| Use LBM instead of Track-On | online point tracking · RoboTAP | AJ: 63.5 -> 61.4 | Hurt |
| Use LBM instead of OVTrack+ | training-free object tracking · OVT-B | TETA (All): +9.2 TETA | Helped |
| Use LBM instead of NetTrack | zero-shot dynamic object tracking · BFT test | OWTA: +2.0 OWTA | Helped |
| Add streaming module (ablation) | LBM · TAP-Vid DAVIS | AJ: 64.2 -> 65.1; 0.9 AJ | Helped |
| Add collision module (ablation) | LBM · TAP-Vid DAVIS | OA: 88.3 -> 89.5 | Helped |
| Use 3 instead of 2 predict-update layers (ablation) | LBM · TAP-Vid DAVIS | AJ: 64.2 -> 65.1; +0.9 AJ | Helped |
| Use 4 instead of 3 predict-update layers (ablation) | LBM · TAP-Vid DAVIS | AJ: 65.1 -> 64.9 | No clear effect |
| Replace ResNet18 with Swin-T (ablation) | LBM · TAP-Vid DAVIS | AJ: 65.1 -> 63.0; 2.1 AJ | Hurt |
| Remove the label penalty term (ablation) | LBM object association · TAO validation | TETA (All): 45.3 -> 44.3 | Hurt |
| Remove detection score weighting while retaining label penalty (ablation) | LBM object association · TAO validation | TETA (All): 45.3 -> 45.2 | No clear effect |
Surprising findings
- A Swin-T encoder underperforms ResNet18 (63.0 vs 65.1 AJ) while being slower and larger.
- Removing both streaming and collision modules still yields 51.8 AJ, with only modest drops from removing either module alone.
Limitations
- Collision-streaming locality causes discontinuity and drift in long-term tracking and on discontinuous video splices.
- Random sampling of pixels within bounding boxes is vulnerable to background interference.
- Tracking fails on visually uniform textures and lacks object-level state recovery after detection failure.
Experimental setup
Datasets: TAP-Vid Kubric ImageNet TAP-Vid DAVIS TAP-Vid Kinetics RoboTAP TAO BFT OVT-B LVIS YTVIS
Benchmarks: TAP-Vid DAVIS TAP-Vid Kinetics RoboTAP TAO OVT-B BFT AnimalTrack
Metrics: AJ delta_avg_x OA TETA OWTA FPS
Baselines: TAPIR LocoTrack BootsTAPIR CoTracker3 CoTracker PIPs PIPs++ TAPTR TAPTRv2 SpatialTracker MFT Online TAPIR
Builds on: lattice Boltzmann method CoTracker Track-On NetTrack deformable attention
Closest papers by meaning
- DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos (NeurIPS 2025 · similarity 0.83)
- TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels (NeurIPS 2025 · similarity 0.83)
- CSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Features (ICML 2025 · similarity 0.82)
- SpatialTracker: Tracking Any 2D Pixels in 3D Space (CVPR 2024 · similarity 0.82)
- DELTA: DENSE EFFICIENT LONG-RANGE 3D TRACKING FOR ANY VIDEO (ICLR 2025 · similarity 0.82)
- HiCoM: Hierarchical Coherent Motion for Dynamic Streamable Scenes with 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.