Research map / Image generation / Editing and style transfer
Do Image Editing Models Understand Lighting?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces 3DLP, a benchmark of 1K real HDR image pairs (light probe on/off) with pixel-level physically motivated metrics (SIE, LFE) that are invariant to white balance and exposure, used to evaluate six image editing models on lighting understanding.
Problem
Existing benchmarks for image editing judge lighting edits via VLMs, human judgement, or synthetic data, which cannot quantify pixel-level light transport accuracy in real indoor scenes.
Approach
Capture real HDR on/off light probe pairs and score edits with scale- and shift-invariant ratio-image metrics (SIE on standardised intensity, LFE on low-frequency gradients) so models may freely adjust exposure, white balance, and lamp colour/intensity.
Key results
- Nano Banana Pro achieves the best average rank (1.0) with SIE 0.813 (turn-on) and 0.769 (turn-off), ahead of Nano Banana 2 (0.882 / 0.871).
- Qwen-Image-Edit is the best open-source model (SIE 1.239 turn-on, 0.982 turn-off), clearly ahead of Flux 2 Dev (1.747 / 2.136).
- VLM A/B tests fail to match the benchmark ranking; e.g., GPT Image 1.5 is ranked last by the benchmark yet scored 65.9-67.6% by VLMs, and Flux 2 Max scores 24.6-28.0% indicating VLMs prefer generated artefacts.
Surprising findings
- A 'Do Nothing' model scores among the best on PSNR, SSIM and LPIPS, showing these metrics fail to penalise failed light toggling.
- VLM judges (Gemini 3.1 Pro, GPT-5.5) disagree with each other and with the physical benchmark ranking, e.g. GPT Image 1.5 ranks last on 3DLP but is favored by VLMs.
Limitations
- Restricted to indoor scenes because the probe's ~2500-lumen output is overpowered by outdoor daylight.
- Task-specific relighting methods could not be evaluated because no code or weights were available.
- Metrics computed on best 80% of images to exclude task misinterpretation; all models still fail on challenging transparent objects and higher-order bounces.
Experimental setup
Datasets: 3DLP real-world HDR dataset (1K image pairs)
Benchmarks: 3DLP benchmark
Metrics: SIE LFE PSNR SSIM LPIPS
Baselines: Nano Banana 2 Qwen-Image-Edit Flux 2 Max Flux 2 Dev GPT Image 1.5 Bagel 7B OmniGen2 PSNR SSIM LPIPS
Builds on: PSNR SSIM LPIPS
Closest papers by meaning
- DiffusionLight: Light Probes for Free by Painting a Chrome Ball (CVPR 2024 · similarity 0.87)
- UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback (NeurIPS 2025 · similarity 0.87)
- ScribbleLight: Single Image Indoor Relighting with Scribbles (CVPR 2025 · similarity 0.86)
- SyncLight: Single-Edit Multi-View Relighting (NeurIPS 2026 · similarity 0.86)
- LightIt: Illumination Modeling and Control for Diffusion Models (CVPR 2024 · similarity 0.86)
- Text2Relight: Creative Portrait Relighting with Text Guidance (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.