Research map / Video / Video segmentation and tracking
Turb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a segment-then-restore pipeline for dynamic videos under atmospheric turbulence, combining cross-correlation stabilization, average-optical-flow motion segmentation, Gaussian-weighted background stacking, and a Restormer trained on a procedural simplex/Perlin-noise turbulence simulator.
Problem
Restoring videos of dynamic scenes degraded by atmospheric turbulence, where object motion and turbulence jointly break optical-flow-based multi-frame registration and existing methods target static or small-motion scenes.
Approach
Separate dynamic foreground from static background via unsupervised average optical flow segmentation, enhance the background with turbulence-strength-adaptive Gaussian-weighted stacking, merge with Poisson blending, and sharpen with a transformer trained on a fast noise-based tilt-and-blur simulator.
Key results
- On CLEAR, the method reaches about 26.5 dB PSNR and 0.85-0.87 SSIM on low/medium turbulence, versus about 22.9-23.4 dB and 0.74-0.77 for AT-Net.
- Average optical flow segmentation achieves mean IoU 0.627 on URG-T, second to RGA (0.696) and above TMO, DSNet and Deformable Sprites.
- Line deviation score of 1.89 vs 3.33 for AT-Net and 5.79 for TurbNet; per-frame latency is 5.71 s at 1080p on an A100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add video stabilization (ablation) | Turb-Seg-Res · CLEAR dataset | PSNR: 26.80 to 28.09 | Helped |
| add video stabilization (ablation) | Turb-Seg-Res · CLEAR dataset | SSIM: 0.888 to 0.935 | Helped |
| add segmentation and background enhancement before transformer restoration (ablation) | Turb-Seg-Res · dynamic video with moving car and static STOP sign | visual restoration quality | Helped |
| use AOF segmentation instead of RGA (ablation) | Turb-Seg-Res segmentation · URG-T | Mean IoU: 0.696 vs 0.627 | Hurt |
Surprising findings
- The paper reports that its segmentation is simpler than RGA yet ranks second, trading a small IoU drop for much lower latency.
Limitations
- Segmentation accuracy is below the best unsupervised method (RGA) on URG-T.
- Noise-based simulator is less physically accurate than physics-based simulators; gap not closed.
- Per-frame latency of about 5.7 s at 1080p limits real-time use.
Experimental setup
Datasets: CLEAR OTIS URG-T Synthetic turbulence clips from procedural simulator
Benchmarks: CLEAR OTIS URG-T
Metrics: PSNR SSIM IoU line deviation (Hough-based) latency
Baselines: AT-Net TCI 2020 TurbNet RGA TMO DS-Net Deformable Sprites P2S simulator
Builds on: RAFT Restormer Poisson image editing Lucky imaging / TCI
Closest papers by meaning
- NB-GTR: Narrow-Band Guided Turbulence Removal (CVPR 2024 · similarity 0.85)
- Temporally Consistent Atmospheric Turbulence Mitigation with Neural Representations (NeurIPS 2024 · similarity 0.85)
- Spatio-Temporal Turbulence Mitigation: A Translational Perspective (CVPR 2024 · similarity 0.85)
- Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation (CVPR 2025 · similarity 0.84)
- Reversing Flow for Image Restoration (CVPR 2025 · similarity 0.84)
- FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.