Research map / Detection and segmentation / Tracking
Projecting Trackable Thermal Patterns for Dynamic Computer Vision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Projects laser-drawn heat patterns that a thermal camera tracks on textureless objects, and trains a ResNet to reverse heat diffusion so standard vision pipelines (SfM, optical flow, tracking) can use them.
Problem
Textureless objects and scenes are hard for visual tracking, SfM and optical flow; physical markers alter appearance and projected patterns do not stick to surfaces across frames.
Approach
Use a co-located thermal camera and steerable laser to paint persistent heat dot patterns (Halton sequence, few dots per frame) that remain trackable in the thermal domain, and train a projection-diffusion reversal network to remove newly projected dots and undo diffusion so frame pairs match brightness-constancy assumptions.
Key results
- On a turntable SfM scan, the recovered camera positions fit a circle with R^2=0.99996 with PDR, versus 0.94 and 0.99 for two cases without PDR.
- Indoor cart localization over roughly 20 m loop gave a mean absolute error of about 14 cm against a GoPro-based reference.
- Scanned textureless plastic head reconstruction reached mean Hausdorff distance 1.5 mm (0.8% of shape scale) against a painted reference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PDR (ablation) | Thermal-pattern SfM on cylindrical can · SfM, cylindrical can | circle-fit R2: 0.94 -> 0.9991 | Helped |
| add PDR (ablation) | Thermal-pattern SfM on rectangular planter · SfM, rectangular planter | circle-fit R2: 0.99 -> 0.999 | Helped |
| add thermal patterns and track their points | SfM on low-albedo, mostly textureless objects · SfM, styrofoam head | circle-fit R2: R2 of 0.99996 | Helped |
| add thermal patterns and use PDR-corrected frames | RAFT optical flow · Optical flow, rotating stage | flow-model R2: R2=0.98 | Helped |
| add PDR (ablation) | Thermal-pattern tracking on slowly diffusing materials · Dynamic vision tasks, slow point diffusion | downstream task performance | No clear effect |
Surprising findings
- The SNR of projected heat dots deteriorates rapidly when more than about two dots are projected per frame, favoring discrete dot patterns over continuous lines.
Limitations
- Degraded performance on materials that are poorly responsive to heat, such as glass and metals.
- Thermal patterns are transient, so feature-based loop closure relies on natural infrared texture; dense MVS reconstruction is not straightforward in the thermal domain.
- Laser safety concerns for humans and animals when using visible green laser; eye-safe mid-IR laser suggested.
Experimental setup
Datasets: self-collected thermal video of 13 static scenes
Metrics: circle-fit R^2 of recovered camera motion mean absolute error vs GoPro reference (cm) Hausdorff distance to reference mesh (mm) SNR of projected dots
Baselines: RAFT without PDR SfM without PDR operator
Builds on: RAFT Lucas-Kanade COLMAP Shi-Tomasi corner detection
Closest papers by meaning
- A Theory of Joint Light and Heat Transport for Lambertian Scenes (CVPR 2024 · similarity 0.80)
- Towards Realistic Scene Generation with LiDAR Diffusion Models (CVPR 2024 · similarity 0.80)
- BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling (AAAI 2024 · similarity 0.80)
- ThermalGaussian: Thermal 3D Gaussian Splatting (ICLR 2025 · similarity 0.80)
- MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Field (ICLR 2024 · similarity 0.80)
- LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulation (CVPR 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.