Research map / Detection and segmentation / CNN features
AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaptiveISP, an actor-critic RL system that greedily selects and parameterizes differentiable ISP modules stage by stage, using a frozen YOLO-v3 detector's loss as reward to produce task-driven, scene-adaptive ISP pipelines.
Problem
Existing ISP tuning methods optimize for image quality rather than downstream recognition, and learned ISP pipelines are fixed at inference, degrading in dynamic scenes; searches for ISP structure are also slow.
Approach
Model ISP configuration as an MDP where a lightweight CNN policy picks one differentiable ISP module and its parameters per stage, rewarded by the reduction in a frozen detector's error, plus penalties for module reuse, entropy, and computational time, so pipeline length adapts per image.
Key results
- On LOD, AdaptiveISP reaches mAP@0.5 of 71.4 and mAP@0.5:0.95 of 47.1, versus 70.9 and 46.6 for the best baseline (Attention-aware Learning).
- With 1.2 ms per stage on a GTX 1660Ti, the policy network adds little inference cost; lambda_c=0.1 reduces per-sample time from 14.73 ms to 9.20 ms at a cost of about 1.5 mAP@0.5.
- Adding the frozen YOLO-v3 signal to DDQ raises mAP@0.5 from 50.3 to 74.0 on LOD, though DDQ was not used in training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace a fixed handcrafted ISP with AdaptiveISP | YOLOv3 object detection · LOD | mAP@0.5: 70.9 -> 71.4 | Helped |
| replace a fixed handcrafted ISP with AdaptiveISP | YOLOv3 object detection · OnePlus | mAP@0.5: 70.9 -> 70.1 | Hurt |
| replace a fixed handcrafted ISP with AdaptiveISP | YOLOv3 object detection · OnePlus | mAP@0.75: 48.9 -> 49.7 | Helped |
| replace a fixed handcrafted ISP with AdaptiveISP | YOLOv3 object detection · Raw COCO | mAP@0.5: 53.8 -> 54.9 | Helped |
| apply AdaptiveISP instead of the RGB input pipeline | YOLOX · LOD | mAP@0.5: 57.0 -> 69.7 | Helped |
| apply AdaptiveISP instead of the RGB input pipeline | DDQ · LOD | mAP@0.5: 50.3 -> 74.0 | Helped |
| add AdaptiveISP before the segmentation model | YOLOv5 segmentation · Raw COCO segmentation | mAP@0.5: 46.4 -> 47.0 | Helped |
| increase computational-cost penalty λc to 1e-2 (ablation) | AdaptiveISP · LOD | total time (ms): 14.73 -> 11.54 | Helped |
| increase computational-cost penalty λc to 1e-2 (ablation) | AdaptiveISP · LOD | mAP@0.5: 71.4 -> 71.0 | Hurt |
| limit the pipeline to 1 stage instead of 5 (ablation) | AdaptiveISP · LOD | mAP@0.5: 71.4 -> 69.6 | Hurt |
| limit the pipeline to 2 stages instead of 5 (ablation) | AdaptiveISP · LOD | mAP@0.5: 71.4 -> 70.9 | No clear effect |
| limit the pipeline to 4 stages instead of 5 (ablation) | AdaptiveISP · LOD | mAP@0.5: 71.4 -> 71.4 | No clear effect |
Surprising findings
- Denoising, commonly considered essential in traditional ISPs, is reported as not crucial for detection performance.
- Fully desaturating color is chosen for high-ISO low-light images to improve detection, a behavior uncommon in image-quality ISPs.
Limitations
- Relies on differentiable ISP modules, excluding non-differentiable ISP methods.
- Training requires a frozen detector and gradient flow through the ISP, and the method only evaluated on sRGB-domain processing after a simple linear raw conversion.
- Some datasets are small (OnePlus has 50 training images) and COCO results use synthetic raw images.
Experimental setup
Datasets: LOD OnePlus COCO UPI-synthesized raw COCO ROD
Benchmarks: LOD OnePlus raw COCO ROD
Metrics: mAP@0.5 mAP@0.75 mAP@0.5:0.95
Baselines: Crafting NeuralAE DynamicISP Hyperparameter Optimization Attention-aware Learning ReconfigISP Refactoring ISP
Builds on: actor-critic Markov Decision Process YOLO-v3 Attention-aware Learning ReconfigISP
Closest papers by meaning
- Learning To Exploit the Sequence-Specific Prior Knowledge for Image Processing Pipelines Optimization (CVPR 2023 · similarity 0.88)
- ISP-Teacher:Image Signal Process with Disentanglement Regularization for Unsupervised Domain Adaptive Dark Object Detection (AAAI 2024 · similarity 0.86)
- RL-SeqISP: Reinforcement Learning-Based Sequential Optimization for Image Signal Processing (AAAI 2024 · similarity 0.86)
- Test-Time Adaptive Object Detection with Foundation Model (NeurIPS 2025 · similarity 0.83)
- Learning Degradation-Independent Representations for Camera ISP Pipelines (CVPR 2024 · similarity 0.83)
- YOLOv10: Real-Time End-to-End Object Detection (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.