Research map / Image generation / Image restoration
Rolling Shutter Correction with Intermediate Distortion Flow Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes directly estimating the GS-to-RS distortion flow with a global correlation-based flow attention and multi-distortion flow decoding, then backward-warping RS features to recover the GS image for rolling shutter correction.
Problem
Rolling shutter correction is limited by inaccurate undistortion flow from RS to GS, since linear or quadratic rescaling of inter-frame flow cannot model complex non-linear, large motions, and forward warping creates artifacts.
Approach
Directly predict the distortion flow from the latent GS frame to the RS frames (rather than undistortion flow), using global correlation attention between GS and RS features for initial flow, coarse-to-fine refinement, and backward warping with multiple flow fields.
Key results
- On Fastec-RS, the 3-frame model reaches 30.00 dB PSNR and 0.882 SSIM versus 29.49 dB for QRSC (5F), at 34 ms runtime.
- On real-world BS-RSC, the 3-frame model achieves 34.48 dB PSNR, about 1 dB above QRSC (5F) at 33.50 dB ACC-subset comparison and 2.5 dB over QRSC (3F).
- The 2-frame model (2.87M params) beats JAMNet on Fastec-RS (28.88 vs 28.70 dB) with about 40% fewer parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| directly estimate distortion flow from the latent GS image to RS frames (ablation) | RSC model · Fastec-RS | PSNR: 28.39 -> 28.88 | Helped |
| directly estimate distortion flow instead of undistortion flow (ablation) | RSC model · Fastec-RS | PSNR: 27.78 -> 28.39 | Helped |
| remove inter-frame motion modeling (ablation) | RSC model · Fastec-RS | PSNR: 28.88 -> 26.88 | Hurt |
| add global correlation-based flow attention (ablation) | RSC model · Fastec-RS | PSNR: 28.52 -> 28.88 | Helped |
| predict 4 distortion fields instead of 1 (ablation) | RSC model · Fastec-RS | PSNR: 28.73 -> 28.88 | Helped |
| predict 8 distortion fields instead of 4 (ablation) | RSC model · Fastec-RS | PSNR: 28.88 -> 28.82 | Hurt |
| use the 3-frame model rather than the 2-frame model | proposed RSC method · Fastec-RS | PSNR: 28.88 -> 30.00 | Helped |
| use the 3-frame model rather than the 2-frame model | proposed RSC method · Carla-RS | PSNR: 31.33 -> 32.10 | Helped |
| use the proposed 3-frame method instead of QRSC (5F) | RSC method · BS-RSC | PSNR: 33.50 -> 34.48 | Helped |
| use the proposed 3-frame method instead of QRSC (5F) | RSC method · ACC | PSNR: 33.36 -> 34.35 | Helped |
| use the proposed 3-frame method instead of QRSC (5F) | RSC method · Carla-RS | SSIM: 0.933 -> 0.930 | Hurt |
Surprising findings
- Direct distortion flow with backward warping outperforms undistortion flow even when both use the same network design (28.39 vs 27.78 dB PSNR in ablation).
Limitations
- Does not fully address real-world RS distortions due to limited dataset scale.
- Requires unknown camera parameters; does not explicitly model exposure parameters.
- Two-frame model struggles because missing content must be hallucinated.
Experimental setup
Datasets: Fastec-RS Carla-RS BS-RSC
Benchmarks: Fastec-RS Carla-RS BS-RSC ACC
Metrics: PSNR SSIM LPIPS runtime parameters
Baselines: DiffSfM DSUN SUNet VideoRS JAMNet JCD AdaRSC QRSC
Builds on: RAFT GMFlow JAMNet AdaRSC DSUN
Closest papers by meaning
- Learning Adaptive Warping for Real-World Rolling Shutter Correction (CVPR 2022 · similarity 0.91)
- Neural Global Shutter: Learn To Restore Video From a Rolling Shutter Camera With Global Reset Feature (CVPR 2022 · similarity 0.91)
- Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes (CVPR 2021 · similarity 0.89)
- Single Image Rolling Shutter Removal with Diffusion Models (AAAI 2025 · similarity 0.87)
- USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields (ICLR 2024 · similarity 0.85)
- Motion Blur Decomposition with Cross-shutter Guidance (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.