Research map / Reinforcement learning / Visual and representation RL
Goal Conditioned Reinforcement Learning for Photo Finishing Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a goal-conditioned TD3-based RL policy that tunes black-box photo finishing pipeline parameters from a goal image in about 10 queries, for photo finishing and photo stylization tuning.
Problem
Automatically tuning parameters of black-box, non-differentiable photo finishing pipelines (e.g., Lightroom, Darktable) to match a goal image, where zeroth-order search is slow and differentiable proxies are inaccurate or hard to train.
Approach
A goal-conditioned RL policy treats the finishing pipeline as a black box and iteratively outputs pipeline parameters from the current image, goal image, and history. Its state combines dual-path CNN features, photo statistics (histograms, luminance, contrast, saturation), and historical action embeddings, trained with TD3 using PSNR-difference or style-based rewards.
Key results
- On FiveK expert-C targets, the method reaches 35.89 dB PSNR in 10 queries versus 28.53 dB for CMA-ES with 200 queries.
- On HDR+ targets, the method reaches 31.54 dB PSNR in 10 queries versus 28.08 dB for CMA-ES with 200 queries.
- At 4K input, the method takes 1.23 s per execution versus 144.02 s for CMA-ES.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use goal-conditioned RL instead of CMAES search | Photo finishing tuning · FiveK-Target (expert-C) | PSNR: 28.53 -> 35.89 | Helped |
| Use goal-conditioned RL instead of CMAES search | Photo finishing tuning · Random-Target | PSNR: 32.29 -> 38.46 | Helped |
| Use goal-conditioned RL instead of CMAES search | Photo finishing tuning · HDR+ Target | PSNR: 28.08 -> 31.54 | Helped |
| Use goal-conditioned RL rather than CMAES search | Photo finishing tuning · FiveK-Target (expert-C) | Queries: 200 -> 10 | Helped |
| Use goal-conditioned RL rather than CMAES search | Photo finishing tuning · Random-Target | Queries: 200 -> 10 | Helped |
| Use goal-conditioned RL rather than CMAES search | Photo finishing tuning · HDR+ Target | Queries: 200 -> 10 | Helped |
| Use goal-conditioned RL rather than Monolithic Proxy | Photo finishing tuning · 720P | Time(s): 7.67 -> 0.27 | Helped |
| Use goal-conditioned RL rather than CMAES | Photo finishing tuning · 4K | Time(s): 144.02 -> 1.23 | Helped |
| Use the proposed RL method rather than the compared baselines | Photo stylization tuning · Adobe Lightroom Discover style-image pairs | User preference | Helped |
| Add dual-path global and local features (ablation) | RL policy state representation · FiveK Random-Target | PSNR | Helped |
| Add photo statistics representation (ablation) | RL policy state representation · FiveK Random-Target | PSNR | Helped |
| Add historical action representation (ablation) | RL policy state representation · FiveK Random-Target | PSNR | Helped |
Limitations
- Supports only image conditions; text or other non-image goals are not handled.
- Policy trained on FiveK only; out-of-distribution evaluation is limited to a few datasets.
- Uses a fixed pipeline of nine parameters from standard operations.
Experimental setup
Datasets: MIT-Adobe FiveK HDR+ Lightroom Discover style images
Benchmarks: MIT-Adobe FiveK validation HDR+ subset
Metrics: PSNR SSIM LPIPS queries runtime
Baselines: CMA-ES Monolithic Proxy Cascaded Proxy Greedy Search
Builds on: TD3 CMA-ES
Closest papers by meaning
- RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward (CVPR 2026 · similarity 0.87)
- Learning to Control Camera Exposure via Reinforcement Learning (CVPR 2024 · similarity 0.86)
- AutoEdit: Automatic Hyperparameter Tuning for Image Editing (NeurIPS 2025 · similarity 0.85)
- PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference (NeurIPS 2024 · similarity 0.84)
- Policy Optimized Text-to-Image Pipeline Design (NeurIPS 2025 · similarity 0.83)
- VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.