Research map / Detection and segmentation / CNN features
RL-SeqISP: Reinforcement Learning-Based Sequential Optimization for Image Signal Processing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates hardware ISP parameter tuning as a sequential RL problem, using a PPO actor-critic agent with a dynamic parameter optimization module driven by downstream task feedback.
Problem
Hardware image signal processor (ISP) parameters, numerous and interdependent, are manually tuned by experts in a time-consuming way biased toward human perception; existing automatic methods are data-hungry or fix parameters per method.
Approach
Treats ISP parameter tuning as a multi-step RL process where an actor-critic agent predicts parameter updates from image and parameter state, and a dynamic weighting module suppresses updates near parameter bounds or at the target metric to avoid local optima, with rewards from frozen downstream task networks.
Key results
- Instance segmentation mAP 0.63 with 1,000 training images vs 0.52 for Attention-aware trained on 10,000 images.
- Object detection mAP 0.64 vs 0.61 for Attention-aware.
- IQA SSIM 0.952 and PSNR 35.228 vs 0.901 and 31.904 for default parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DPOM (ablation) | RL-SeqISP · MSCOCO validation, instance segmentation | mAP: 0.54 -> 0.58 | Helped |
| Add DPOM (ablation) | RL-SeqISP · MSCOCO validation, object detection | mAP: 0.55 -> 0.64 | Helped |
| Add DPOM (ablation) | RL-SeqISP · IQA | SSIM: 0.93 -> 0.95 | Helped |
| Add DPOM (ablation) | RL-SeqISP · IQA | PSNR: 33.50 -> 35.23 | Helped |
| Add DPFFM (ablation) | RL-SeqISP · MSCOCO validation, instance segmentation | mAP: 0.52 -> 0.58 | Helped |
| Add DPFFM (ablation) | RL-SeqISP · MSCOCO validation, object detection | mAP: 0.55 -> 0.64 | Helped |
| Add DPFFM (ablation) | RL-SeqISP · IQA | SSIM: 0.93 -> 0.95 | Helped |
| Add DPFFM (ablation) | RL-SeqISP · IQA | PSNR: 33.70 -> 35.23 | Helped |
| Set maximum optimization steps to T=8 instead of T=4 (ablation) | RL-SeqISP · MSCOCO validation, instance segmentation | mAP: 0.58 -> 0.63 | Helped |
| Set maximum optimization steps to T=8 instead of T=4 (ablation) | RL-SeqISP · MSCOCO validation, object detection | mAP: 0.64 -> 0.53 | Hurt |
| Set maximum optimization steps to T=8 instead of T=4 (ablation) | RL-SeqISP · IQA | SSIM: 0.95 -> 0.94 | Hurt |
| RL-SeqISP rather than Attention-aware ISP tuning | MSCOCO validation, instance segmentation · MSCOCO validation, instance segmentation | mAP: 0.52 -> 0.63 | Helped |
Surprising findings
- Performance is non-monotonic in the max optimization step T; larger T (8 or 16) degraded object detection and IQA results.
Limitations
- IQA dataset is small (252 images) and collected with a single sensor.
- Segmentation and detection rely on simulated RAW data from MSCOCO rather than real RAW-RGB pairs.
- Expert tuning takes about 12 hours per image, so the IQA reference is costly to produce.
Experimental setup
Datasets: MSCOCO IMX766 RAW-RGB dataset (252 images collected)
Benchmarks: MSCOCO IMX766 IQA dataset
Metrics: mAP SSIM PSNR
Baselines: Default parameters Expert-tuned Blockwise-tuned Hardware-tuned Attention-aware
Builds on: PPO ResNet
Closest papers by meaning
- Learning To Exploit the Sequence-Specific Prior Knowledge for Image Processing Pipelines Optimization (CVPR 2023 · similarity 0.89)
- AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection (NeurIPS 2024 · similarity 0.86)
- Q-Insight: Understanding Image Quality via Visual Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO (NeurIPS 2025 · similarity 0.84)
- RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression (CVPR 2025 · similarity 0.84)
- DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.