Research map / Reinforcement learning / Policy gradient and actor-critic
Stabilizing and Accelerating Autofocus with Expert Trajectory Regularized Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an actor-critic DRL autofocus model initialized from a relative-movement single-step MobileNetV2 predictor and regularized by an offline expert trajectory dataset to reduce focus hunting and improve multi-step focusing accuracy.
Problem
Learning-based autofocus predicts focus accurately per step but multi-step lens trajectories often exhibit focus hunting (back-and-forth lens movement), slowing focusing and reducing accuracy.
Approach
Formulate autofocus as a multi-step MDP solved by PPO actor-critic, initialize the actor from a single-step MobileNetV2 predictor retrained to output relative lens movement, and regularize the actor with an imitation loss on expert trajectories generated from prior knowledge that avoid focus hunting.
Key results
- Single-step exact (=0) accuracy 0.336 vs 0.243 for RoI-PE & Lens-PE [7] and 0.330 for [7] with relative label.
- 2-step focus hunting rate 0.170 for expert-trajectory DRL vs 0.275 for [7] baseline; 4-step MAE 1.694 vs 2.442.
- Expert regularization is critical: vanilla PPO without it reaches FH rate ~0.99 and much lower accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add expert trajectory regularization to the DRL policy (ablation) | DRL autofocus · AF dataset test, 2-step | exact prediction (= 0): 0.029 -> 0.439 | Helped |
| Add expert trajectory regularization to the DRL policy (ablation) | DRL autofocus · AF dataset test, 2-step | FH rate: 0.992 -> 0.170 | Helped |
| Use relative movement labels instead of absolute lens-position labels (ablation) | AF prediction network · AF dataset test, single-step | exact prediction (= 0): 0.243 -> 0.330 | Helped |
| Use relative movement labels instead of absolute lens-position labels (ablation) | AF prediction network · AF dataset test, single-step | exact prediction (= 0): 0.258 -> 0.326; 0.226 -> 0.305 | Helped |
| Use relative movement labels instead of absolute lens-position labels (ablation) | MobileNetV2 AF prediction network · AF dataset test, 2-step | exact prediction (= 0): 0.275 -> 0.413 | Helped |
| Add DRL with expert trajectory regularization to the relative-label policy | [7] with relative label · AF dataset test, 2-step | exact prediction (= 0): 0.413 -> 0.439 | Helped |
| Add DRL with expert trajectory regularization to the relative-label policy | [7] with relative label · AF dataset test, 2-step | FH rate: 0.261 -> 0.170 | Helped |
| Add DRL with expert trajectory regularization to the relative-label policy | [7] with relative label · AF dataset test, 4-step | exact prediction (= 0): 0.423 -> 0.449 | Helped |
| Add DRL with expert trajectory regularization to the relative-label policy | [7] with relative label · AF dataset test, 4-step | FH rate: 0.358 -> 0.178 | Helped |
| Use Expert Trajectory instead of RoI-PE & Lens-PE [7] | AF autofocus policy · AF dataset test, average steps to stop at = 0 | average steps: 6.651 -> 5.014 | Helped |
Surprising findings
- Vanilla DRL without expert regularization performs very poorly, with focus hunting rates near 0.99, due to the large discrete action space (97 actions).
Limitations
- Baselines [7] and [16] were reproduced because no open-source code was available.
- Failure cases include textureless scenes, small targets with complex backgrounds, and horizontal structures.
- Evaluated only on a single dataset captured with one smartphone model.
Experimental setup
Datasets: Herrmann et al. AF dataset (Google Pixel 3 dual-pixel) expert trajectory dataset (generated)
Benchmarks: Herrmann et al. AF dataset
Metrics: accuracy at 0/<=1/<=2/<=4 focal indices MAE RMSE focus hunting rate average steps to stop
Baselines: SSD Disparity ZNCC Disparity with Calibration Learning the Depth Learning to Autofocus RoI-PE & Lens-PE DRL in [6] DRL in [54]
Builds on: PPO RoI-PE & Lens-PE (Choi et al.) Learning to Autofocus (Herrmann et al.) MobileNetV2
Closest papers by meaning
- Focus-Then-Decide: Segmentation-Assisted Reinforcement Learning (AAAI 2024 · similarity 0.84)
- One-Step Event-Driven High-Speed Autofocus (CVPR 2025 · similarity 0.83)
- LENS: Learning to Segment Anything with Unified Reinforced Reasoning (AAAI 2026 · similarity 0.82)
- Learning to Control Camera Exposure via Reinforcement Learning (CVPR 2024 · similarity 0.82)
- RL-SeqISP: Reinforcement Learning-Based Sequential Optimization for Image Signal Processing (AAAI 2024 · similarity 0.82)
- Goal Conditioned Reinforcement Learning for Photo Finishing Tuning (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.