Research map / Video / Video segmentation and tracking
Deep Video Inverse Tone Mapping Based on Temporal Clues
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a video inverse tone mapping pipeline that uses global sampling of distant reference frames and local propagation of key-frame features to recover over-exposed HDR regions with temporal consistency, trained on a synthesized HDR video dataset.
Problem
Inverse tone mapping of LDR video to HDR: single-frame image ITM methods ignore temporal information and cause flicker, while existing video ITM methods struggle to find and use distant temporal clues for recovering over-exposed regions.
Approach
A Global Sampling and Local Propagation strategy: sparse key frames are reconstructed by aligning distant reference frames (temporal clues) via an Incremental Clue Aggregation Module (flow-correction, flow-guided deformable convolution, swin transformer global aggregation), then key-frame features are propagated to neighbors by a bidirectional Feature and Clue Propagation Module.
Key results
- Proposed method reaches HDR-VDP-3 of 7.891 on REDS-val and HDR-VQM of 0.398 on HDM-HDR, beating the best baseline BasicVSR++ (7.254 and 0.443 respectively).
- On MPI-HDRv the proposed method scores HDR-VQM 0.037 versus 0.049 for Deep VITM.
- Running time is 521 ms per 512x512 frame with the full pipeline versus 1236 ms when using only ICAM on every frame (Tesla V100).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed temporal-clue video ITM method instead of BasicVSR++ | video ITM · REDS-val | HDR-VDP-3: 7.254 -> 7.891 | Helped |
| use the proposed temporal-clue video ITM method instead of BasicVSR++ | video ITM · REDS-val | HDR-VQM: 0.443 -> 0.398 | Helped |
| use the proposed temporal-clue video ITM method instead of the strongest listed baseline | video ITM · HDM-HDR | HDR-VDP-3: 6.346 -> 6.754 | Helped |
| use the proposed temporal-clue video ITM method instead of BasicVSR++ | video ITM · HDM-HDR | HDR-VQM: 0.427 -> 0.356 | Helped |
| use the proposed temporal-clue video ITM method instead of BasicVSR++ | video ITM · LiU-HDR | HDR-VDP-3: 8.265 -> 8.591 | Helped |
| use the proposed temporal-clue video ITM method instead of BasicVSR++ | video ITM · LiU-HDR | HDR-VQM: 0.547 -> 0.522 | Helped |
| use the proposed temporal-clue video ITM method instead of the strongest listed baseline | video ITM · MPI-HDRv | HDR-VDP-3: 7.225 -> 7.970 | Helped |
| use the proposed temporal-clue video ITM method instead of Deep VITM | video ITM · MPI-HDRv | HDR-VQM: 0.049 -> 0.037 | Helped |
| add flow correction convolution (ablation) | ICAM · HDM-HDR | HDR-VQM: 0.396 -> 0.380 | Helped |
| add flow-guided deformable convolution (ablation) | ICAM · HDM-HDR | HDR-VQM: 0.380 -> 0.367 | Helped |
| add global feature aggregation (ablation) | ICAM · HDM-HDR | HDR-VQM: 0.367 -> 0.356 | Helped |
| add alignment components (ablation) | ICAM · HDM-HDR | HDR-VQM: 0.418 -> 0.356 | Helped |
Surprising findings
- Image ITM baseline Single HDR outperforms the earlier video ITM method Deep VITM on several datasets in HDR-VDP-3, and BasicVSR++ is a strong video baseline.
Limitations
- Relies on frozen pre-trained optical flow (GMA) which is inaccurate for over-exposed regions and large luminance differences.
- Trained only on synthetic HDR video data built from LDR video and HDR image datasets, with a noted dynamic-range gap to real HDR content.
- Real-world evaluation uses LDR videos simulated from HDR datasets rather than true captured LDR footage, except one qualitative iPhone example.
Experimental setup
Datasets: SICE REDS HDM-HDRv LiU-HDRv MPI-HDRv
Benchmarks: REDS-val HDM-HDR LiU-HDR MPI-HDRv
Metrics: HDR-VDP-3 HDR-VQM
Baselines: HDRCNN Differentiable HDR Single HDR FMNet Deep VITM BasicVSR++
Builds on: BasicVSR++ GMA optical flow Deformable convolution Swin transformer BasicVSR
Closest papers by meaning
- Zero-Shot Structure-Preserving Diffusion Model for High Dynamic Range Tone Mapping (CVPR 2024 · similarity 0.83)
- A Video is Worth 256 Bases: Spatial-Temporal Expectation-Maximization Inversion for Zero-Shot Video Editing (CVPR 2024 · similarity 0.82)
- Towards Real-World HDR Video Reconstruction: A Large-Scale Benchmark Dataset and A Two-Stage Alignment Network (CVPR 2024 · similarity 0.82)
- One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution (NeurIPS 2025 · similarity 0.81)
- Re-thinking Temporal Search for Long-Form Video Understanding (CVPR 2025 · similarity 0.81)
- VISTA: Enhancing Long-Duration and High-Resolution Video Understanding by Video Spatiotemporal Augmentation (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.