Research map / 3D vision / Human pose estimation
Towards a Perceptual Evaluation Framework for Lighting Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Runs a psychophysical study comparing lighting estimation methods via human 2AFC judgments on virtual object insertion, shows popular IQA metrics disagree with humans, and learns an SVR combination of 15 IQA metrics that better predicts preference.
Problem
Lighting estimation is evaluated with image quality metrics against ground truth renders, but it is unclear whether these metrics match human preference when estimated lighting is used to relight virtual objects in real photographs.
Approach
Collect human 2AFC preference data across accuracy (matching reference) and plausibility (realism in context) tasks for diffuse and glossy spheres, measure agreement of IQA metrics with humans, and learn an SVR that combines metric differences to predict human preference.
Key results
- Learned SVR metric combination achieves Spearman's rho of 0.689-0.753 across indoor experiments, versus at most 0.529 for the next best single metric (LPIPS for task 1 diffuse).
- Most IQA metrics perform at chance level on task 2 and task 1 glossy; only si-RMSE, SSIM, LPIPS, and FLIP exceed the expected observer agreement on task 1 diffuse indoor.
- On new methods the learned metric reaches agreement 0.786/0.856 (task 1 diffuse/glossy) and 0.787/0.889 (task 2), exceeding the best standard metrics (0.670/0.809 and 0.686/0.691).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a learned combination of existing IQA metrics instead of a single metric | Lighting-estimation evaluation against observer preferences · Indoor, Task 1 and Task 2; diffuse and glossy | Spearman’s ρ: LPIPS 0.529 -> Ours 0.689; RGB Ang. Err. 0.226 -> Ours 0.724; BRISQUE 0.145 -> Ours 0.711; HyperIQA 0.371 -> Ours 0.753 | Helped |
| Use a learned combination of IQA metrics | Evaluation on new indoor lighting-estimation methods · Task 1, diffuse | agreement score: 0.670/0.809 and 0.686/0.691 | Helped |
| Use a learned combination of IQA metrics | Evaluation on new indoor lighting-estimation methods · Task 1 and Task 2, diffuse and glossy | agreement score: 0.786/0.856 for task 1 and 0.787/0.889 for task 2 | Helped |
| Evaluate lighting with a single existing IQA metric | Human preference on lighting-estimation renders · Task 1, diffuse | agreement score | Mixed |
| Use parametric light predictions alone rather than HDR lighting and plausible textures | Gardner et al. indoors and Zhang et al. outdoors · Glossy-object experiments | human preference | Hurt |
| Use a simple image-projection lighting estimate rather than recent methods | Khan et al. · Task 2, outdoor diffuse | human preference | Helped |
| Use a simple image-projection lighting estimate rather than recent methods | Khan et al. · Task 2, indoor and outdoor glossy | human preference | No clear effect |
Surprising findings
- Khan et al.'s simple non-learning method matches recent learned methods in plausibility judgments and even outperforms them for outdoor diffuse scenes.
- Methods that predict only parametric lights perform well on diffuse accuracy but are disliked when glossy objects are rendered.
Limitations
- Virtual object is limited to a simple sphere on a plane, so results may not transfer to complex objects.
- Outdoor study uses only 12 observers and 3 methods, leaving outdoor uncertainty higher.
- Learned metric correlations remain below the 0.8 reliability threshold.
Experimental setup
Datasets: HDR panorama datasets (scenes clustered by spherical harmonics) collected psychophysical perceptual data
Benchmarks: psychophysical study with 49 observers held-out indoor method (Khan et al.) second study with 6 observers on new methods
Metrics: agreement score Spearman's rho Kendall's tau Thurstone z-score
Baselines: Weber et al. 2022 EverLight StyleLight Gardner et al. 2019 Zhang et al. 2019 Khan et al. 2016 RMSE SSIM LPIPS PieAPP FLIP HDR-VDP3
Builds on: Thurstone Case V law of comparative judgement LPIPS HDR-VDP3 Cycles renderer
Closest papers by meaning
- Do Image Editing Models Understand Lighting? (NeurIPS 2026 · similarity 0.85)
- Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered (NeurIPS 2026 · similarity 0.85)
- DreamLight: Towards Harmonious and Consistent Image Relighting (NeurIPS 2025 · similarity 0.84)
- Multidimensional Observer Model and Perceptual Dimensions of Human Image Quality Assessment (NeurIPS 2026 · similarity 0.84)
- Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonization (CVPR 2025 · similarity 0.83)
- Relightful Harmonization: Lighting-aware Portrait Background Replacement (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.