Research map / Image generation / Image restoration
UltraFusion: Ultra High Dynamic Imaging using Exposure Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UltraFusion, an exposure fusion method that treats merging over- and under-exposed images (up to 9 stops apart) as guided inpainting built on Stable Diffusion with a decompose-and-fuse control branch and fidelity branch.
Problem
Standard exposure fusion handles only about 3-4 stops of exposure difference and fails under large exposure gaps, misalignment, lighting variation, and tone-mapping issues in dynamic HDR scenes.
Approach
Model exposure fusion as guided inpainting: the over-exposed image is the base to be inpainted in highlights, while the under-exposed image (pre-aligned, occlusion-masked, decomposed into structure and color) serves as soft guidance within a diffusion model, with a fidelity branch for VAE texture preservation.
Key results
- On the static MEFB dataset, UltraFusion reaches MUSIQ 68.82 versus 66.76 for HSDS-MEF, and the best DeQA-Score (3.881) and PAQ2PIQ/HyperIQA among compared methods.
- On dynamic RealHDRV, UltraFusion achieves TMQI 0.8925 versus 0.8726 for SAFNet (best HDR reconstruction baseline) and 0.8323 for HSDS-MEF.
- Removing the alignment strategy drops RealHDRV TMQI from 0.8925 to 0.7427, the largest drop among the ablated components.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the alignment strategy (pre-alignment and its training data synthesis pipeline) (ablation) | UltraFusion · RealHDRV | TMQI: 0.7427 -> 0.8925 | Helped |
| Add the decompose-and-fuse control branch instead of vanilla ControlNet (ablation) | UltraFusion · RealHDRV | TMQI: 0.8872 -> 0.8925 | Helped |
| Add the fidelity control branch (ablation) | UltraFusion · RealHDRV | TMQI: 0.8763 -> 0.8925 | Helped |
| Use the RGB under-exposed image as guidance instead of decomposed structure and color (ablation) | UltraFusion decompose-and-fuse control branch · RealHDRV | detail retention | Hurt |
| Use structure information alone as guidance instead of structure and color (ablation) | UltraFusion decompose-and-fuse control branch · RealHDRV | color vividness | Hurt |
| Combine under-exposed structure and color information as guidance (ablation) | UltraFusion decompose-and-fuse control branch · RealHDRV | detail and color consistency | Helped |
| Add multi-scale cross-attention to the decompose-and-fuse control branch (ablation) | UltraFusion · RealHDRV | fusion quality | Helped |
| Add pre-alignment at test time (ablation) | UltraFusion alignment strategy · dynamic exposure fusion | alignment robustness | Helped |
| Add the data synthesis pipeline (ablation) | UltraFusion alignment strategy · dynamic exposure fusion | robustness to unaligned conditions | Helped |
Surprising findings
- Naively injecting the RGB under-exposed image as guidance causes the model to ignore details; using only its normalized luminance-free structure plus color works better.
- Without alignment strategy and data synthesis, the model degrades to a standard multi-exposure fusion method that fails on motion.
Limitations
- Occlusion mask estimation can err under extreme exposure differences and non-rigid motion, degrading highlight restoration to single-image HDR.
- Inference takes about 3.3 s per 512x512 pair on an RTX 4090.
- Relies on RAFT optical flow, which the authors note is not robust to large exposure differences.
Experimental setup
Datasets: SICE Vimeo-90K MEFB RealHDRV
Benchmarks: MEFB RealHDRV UltraFusion benchmark
Metrics: MUSIQ DeQA-Score PAQ2PIQ HyperIQA MEF-SSIM TMQI user study voting rate
Baselines: HDR-Transformer SCTNet SAFNet Deepfuse MEF-GAN U2Fusion Defusion MEFLUT HSDS-MEF TC-MoA ControlNet
Builds on: ControlNet Stable Diffusion RAFT HDR-Transformer
Closest papers by meaning
- UltraLED: Learning to See Everything in Ultra-High Dynamic Range Scenes (NeurIPS 2025 · similarity 0.89)
- Single Shot HDR Recovery via a Video Diffusion Prior (NeurIPS 2026 · similarity 0.86)
- Ultra-High-Definition Dynamic Multi-Exposure Image Fusion via Infinite Pixel Learning (AAAI 2025 · similarity 0.86)
- Color Event Enhanced Single-Exposure HDR Imaging (AAAI 2024 · similarity 0.86)
- GaussHDR: High Dynamic Range Gaussian Splatting via Learning Unified 3D and 2D Local Tone Mapping (CVPR 2025 · similarity 0.85)
- High Dynamic Range Novel View Synthesis with Single Exposure (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.