Research map / 3D vision / Depth and stereo
Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SelfHDR, a self-supervised multi-exposure HDR reconstruction method that supervises a reconstruction network with color and structure components built from real dynamic multi-exposure images, without HDR ground truth.
Problem
Deghosting multi-exposure HDR reconstruction in dynamic scenes normally requires HDR ground-truth, which is costly to collect; prior self-supervised methods rely on synthetic pseudo-pairs that generalize poorly to real images.
Approach
Decompose the latent HDR ground truth into a color component (fused from optical-flow-aligned multi-exposure images) and a structure component (from a structure-focused network supervised by the color component and the medium-exposure reference), then train the reconstruction network with masked losses on these two components.
Key results
- SelfHDR with FSHDR backbone reaches 43.80 dB PSNR-u, a 1.58 dB gain over the self-supervised FSHDR K=0 (42.17 dB) with the same network.
- SelfHDR with SCTNet reaches 43.95 dB PSNR-u and 0.9907 SSIM-u, close to fully supervised SCTNet (44.48 dB) and HDR-Transformer (44.21 dB).
- Pre-alignment with optical flow for the color component is crucial; ablation shows PSNR-u drops from 43.68 to 35.50 dB when removed from both components.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SelfHDR instead of FSHDR K=0 with the same reconstruction network | FSHDR reconstruction network · Kalantari et al. test | PSNR-u: +1.58 dB | Helped |
| supervise reconstruction with both color and structure components rather than the color component alone (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u: 34.45 -> 43.68 | Helped |
| supervise reconstruction with both color and structure components rather than the structure component alone (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u: 43.38 -> 43.68 | Helped |
| replace learned reconstruction with direct fusion of the color and structure components (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u | Hurt |
| refine the structure component by using the reconstruction output as new supervision (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u | No clear effect |
| use both structure-expansion and structure-preserving losses instead of either one alone (ablation) | SelfHDR structure-focused network · Kalantari et al. test | PSNR-u | Helped |
| mask harmful regions in structure-preserving and structure-expansion losses (ablation) | SelfHDR structure-focused network · Kalantari et al. test | PSNR-u | Helped |
| design the color-loss mask using the structure component rather than the alternative mask (ablation) | SelfHDR reconstruction network · Kalantari et al. test | PSNR-u | Helped |
| add optical-flow pre-alignment when constructing color supervision (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u: 35.50 -> 43.41 | Helped |
| add optical-flow pre-alignment when constructing structure supervision (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u: 43.41 -> 43.68; 0.27 dB PSNR gain | Helped |
| replace Liu et al. optical flow with PWC-Net (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u: 43.68 -> 43.45 | Hurt |
| replace Liu et al. optical flow with FlowFormer (ablation) | SelfHDR-AHDRNet · Kalantari et al. test | PSNR-u: 43.68 -> 43.50 | Hurt |
Surprising findings
- Re-training the reconstruction network with the refined output as supervision did not improve performance, suggesting the structure component is already sufficient.
- Simply fusing the color and structure components gave best results when the color mask was zero, i.e. no benefit from naive fusion.
Limitations
- Requires clear, noise-free and blur-free input images; noise in short exposures and blur in long exposures are not handled.
- Fails when scene irradiance changes drastically across the multi-exposure shots, making color components inaccurate.
Experimental setup
Datasets: Kalantari et al. dataset Sen et al. dataset Tursun et al. dataset
Benchmarks: Kalantari et al. dataset Sen et al. dataset Tursun et al. dataset
Metrics: PSNR-u SSIM-u PSNR-l SSIM-l HDR-VDP-2
Baselines: AHDRNet FSHDR HDR-Transformer SCTNet FSHDR K=0 Nazarczuk et al. SAME PWC-Net FlowFormer
Builds on: AHDRNet FSHDR HDR-Transformer SCTNet Kalantari et al. HDR deghosting
Closest papers by meaning
- Single Shot HDR Recovery via a Video Diffusion Prior (NeurIPS 2026 · similarity 0.89)
- High Dynamic Range Novel View Synthesis with Single Exposure (ICML 2025 · similarity 0.88)
- Color Event Enhanced Single-Exposure HDR Imaging (AAAI 2024 · similarity 0.87)
- GaussHDR: High Dynamic Range Gaussian Splatting via Learning Unified 3D and 2D Local Tone Mapping (CVPR 2025 · similarity 0.87)
- EvHDR-GS: Event-guided HDR Video Reconstruction with 3D Gaussian Splatting (AAAI 2025 · similarity 0.87)
- Towards Real-World HDR Video Reconstruction: A Large-Scale Benchmark Dataset and A Two-Stage Alignment Network (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.