Research map / Reinforcement learning / Visual and representation RL

Autonomous Goal Detection and Cessation in Reinforcement Learning: A Case Study on Source Term Estimation

AAAI 2025 · Reinforcement learning › Visual and representation RL · approach: In-context and adaptive RL · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AGDC, a particle-filter-based self-feedback module that lets RL agents (DQN, PPO, DDPG) detect and stop at goal completion in Source Term Estimation, outperforming infotaxis, entrotaxis, DCEE and random search.

Problem

Reinforcement learning agents struggle to autonomously detect task completion and stop when no clear environmental reward or termination signal exists, as in gas Source Term Estimation.

Approach

Use a particle filter belief over source parameters and trigger a self-generated reward and episode termination when the standard deviation of the belief falls below a cessation threshold, wrapping any RL algorithm.

Key results

• AGDC-PPO reached 95.27% success rate at 4000 particles, versus 88.39% for Infotaxis at the same particle count.
• AGDC-PPO achieved 99.13% success rate at cessation threshold 0.9; Infotaxis reached roughly 99% only at the highest thresholds but with much longer distances.
• AGDC-DDPG mean traveled distance fell from 34.33 at 100 particles to 16.83 at 4000 particles.

Measured effects reported in the paper

Surprising findings

• Very high cessation thresholds can inflate success rate and reduce distance by making the task criterion less rigorous, so the authors recommend moderate thresholds.
• Random action selection stays below 5% success regardless of particle count.

Limitations

• Evaluation only in a simulated Gaussian diffusion environment with fixed sensor noise parameters.
• Performance is sensitive to the cessation threshold; high thresholds can misleadingly inflate results.
• Baseline statistical methods are reimplemented without detailed hyperparameter reporting; no real-world deployment.

Experimental setup

Datasets: simulated STE environment (Gaussian diffusion plus sensor noise)

Benchmarks: STE simulated environment with 1,000 randomized test scenarios

Metrics: success rate mean traveled distance search time

Baselines: Infotaxis Entrotaxis DCEE Random action selection

Builds on: DQN PPO DDPG Infotaxis Entrotaxis Dual Control for Exploitation and Exploration

Closest papers by meaning

• Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control (ICLR 2025 · similarity 0.87)
• Improving the Continuity of Goal-Achievement Ability via Policy Self-Regularization for Goal-Conditioned Reinforcement Learning (ICML 2025 · similarity 0.87)
• Learning World Models for Unconstrained Goal Navigation (NeurIPS 2024 · similarity 0.86)
• Goal-Conditioned On-Policy Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
• Flattening Hierarchies with Policy Bootstrapping (NeurIPS 2025 · similarity 0.86)
• Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration (NeurIPS 2026 · similarity 0.86)