Research map / Video / Video segmentation and tracking
Decoupling Degradations with Recurrent Network for Video Restoration in Under-Display Camera
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D²RNet, a multi-scale bi-directional recurrent video restoration network with decoupling attention modules that split under-display camera (UDC) video degradation into flare and haze, and introduces the VidUDC33K benchmark.
Problem
Video restoration for under-display camera (UDC) systems, where diffraction from OLED pixel arrays causes flare in bright regions and haze in darker regions that vary with light intensity and motion over time.
Approach
A soft mask generation function splits each frame into flare and haze regions based on max channel intensity; flare is removed using long-term recurrent features and haze using short-term features, inside a multi-scale bi-directional recurrent network with intermediate supervision.
Key results
- On VidUDC33K, D²RNet reaches 31.91 dB PSNR, 0.9313 SSIM and 0.1306 LPIPS, versus 30.89 dB for RVRT.
- Its PSNR gain over RVRT is 1.02 dB, with 0.44 s runtime and 5.76M parameters versus RVRT's 1.68 s and 13.9M.
- Adding intermediate supervision, the haze and flare removal components, and the soft mask raises PSNR from a 30.49 dB base to 31.91 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use D²RNet instead of RVRT | UDC video restoration · VidUDC33K | PSNR: 30.89 -> 31.91; 1.02 dB | Helped |
| use D²RNet instead of RVRT | UDC video restoration · VidUDC33K | SSIM: 0.9261 -> 0.9313 | Helped |
| use D²RNet instead of RVRT | UDC video restoration · VidUDC33K | LPIPS: 0.1314 -> 0.1306 | Helped |
| add intermediate supervision (ablation) | Base D²RNet · VidUDC33K | PSNR: 30.49 -> 31.01 | Helped |
| add short-term haze removal component (ablation) | D²RNet with intermediate supervision · VidUDC33K | PSNR: 31.01 -> 31.25 | Helped |
| add long-term flare removal component (ablation) | D²RNet with intermediate supervision · VidUDC33K | PSNR: 31.01 -> 31.37 | Helped |
| add soft mask generation function (ablation) | D²RNet with intermediate supervision, haze removal, and flare removal · VidUDC33K | PSNR: 31.74 -> 31.91 | Helped |
| vary soft mask threshold to τ=0.98 (ablation) | D²RNet · VidUDC33K | PSNR: 31.91 -> 31.30 | Hurt |
| vary soft mask threshold to τ=0.95 (ablation) | D²RNet · VidUDC33K | PSNR: 31.91 -> 31.74 | Hurt |
| vary soft mask threshold to τ=0.85 (ablation) | D²RNet · VidUDC33K | PSNR: 31.91 -> 31.78 | Hurt |
| use only the 2× scale rather than all three scales (ablation) | D²RNet · VidUDC33K | PSNR: 30.45 -> 31.91 | Hurt |
| add the 4× scale to the 2× scale (ablation) | D²RNet · VidUDC33K | PSNR: 30.45 -> 31.36 | Helped |
Surprising findings
- Image-based UDC restoration (AlignFormer) with fewer parameters falls behind video-specific methods, showing temporal information matters for UDC video.
Limitations
- Relies on simulated degradations from a single measured PSF (ZTE Axon 20), so generality to other devices is unverified.
- Real-data evaluation is only qualitative, on 10 captured videos.
Experimental setup
Datasets: VidUDC33K
Benchmarks: VidUDC33K
Metrics: PSNR SSIM LPIPS
Baselines: DISCNet BNUDC UDC-UNet AlignFormer EDVR FastDVDnet ESTRNN VRT RVRT
Builds on: RVRT UNet SPyNet supervised attention
Closest papers by meaning
- Visual-Instructed Degradation Diffusion for All-in-One Image Restoration (CVPR 2025 · similarity 0.85)
- Degradation-Aware Feature Perturbation for All-in-One Image Restoration (CVPR 2025 · similarity 0.85)
- AverNet: All-in-one Video Restoration for Time-varying Unknown Degradations (NeurIPS 2024 · similarity 0.84)
- Correlation Matching Transformation Transformers for UHD Image Restoration (AAAI 2024 · similarity 0.84)
- DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor (NeurIPS 2024 · similarity 0.84)
- DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.