Research map / Reinforcement learning / Visual and representation RL
Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains an end-to-end SAC agent with multi-scale frontier-based egocentric maps, a scale-grouped CNN, and a total-variation reward to learn online coverage paths in unknown environments for exploration and lawn mowing.
Problem
Online coverage path planning in unknown environments, where the agent must map and cover free space with continuous control under varying coverage definitions (exploration vs. lawn mowing).
Approach
Continuous end-to-end RL policy over multi-scale egocentric coverage, obstacle and frontier maps (frontiers keep non-covered info at coarse scales), with scale-grouped convolutions and a local total-variation reward that discourages small uncovered holes.
Key results
- On Explore-Bench omnidirectional exploration, total T90 of 556 s and T99 of 1061 s, versus 967 s and 1559 s for the best frontier baseline (potential field) on T90/T99 respectively.
- On lawn mowing, total T90 of 310 min and T99 of 409 min, beating the online TSP baseline (448/498) and even the offline TSP on T90 (325), though BSA offline is better (229/270).
- Incremental TV reward raises lawn mowing coverage at 1500 s from 85.1% to 97.8% in ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use end-to-end RL control instead of the strongest frontier baseline | Explore-Bench · Explore-Bench total | T90 (seconds): 967 -> 556 | Helped |
| Use end-to-end RL control instead of the strongest frontier baseline | Explore-Bench · Explore-Bench total | T99 (seconds): 1523 -> 1061 | Helped |
| Use online RL coverage planning instead of online TSP | Lawn mowing · Lawn mowing total, online | T90 (minutes): 448 -> 310 | Helped |
| Use online RL coverage planning instead of online TSP | Lawn mowing · Lawn mowing total, online | T99 (minutes): 498 -> 409 | Helped |
| Add incremental TV reward (ablation) | SGCNN · Lawn mowing | coverage (%): 85.1 -> 97.8 | Helped |
| Add incremental TV reward (ablation) | SGCNN · Exploration | coverage (%): 91.5 -> 93.0 | Helped |
| Add global TV reward (ablation) | SGCNN with incremental TV reward · Lawn mowing | coverage (%): 97.8 -> 96.6 | Hurt |
| Add global TV reward (ablation) | SGCNN with incremental TV reward · Exploration | coverage (%): 93.0 -> 89.0 | Hurt |
| Use CNN instead of MLP (ablation) | Coverage-planning agent · Lawn mowing | coverage (%): 81.4 -> 93.2 | Helped |
| Use CNN instead of MLP (ablation) | Coverage-planning agent · Exploration | coverage (%): 27.3 -> 88.4 | Helped |
| Use scale-grouped convolutions instead of naive CNN (ablation) | Coverage-planning agent · Lawn mowing | coverage (%): 93.2 -> 97.8 | Helped |
| Use scale-grouped convolutions instead of naive CNN (ablation) | Coverage-planning agent · Exploration | coverage (%): 88.4 -> 93.0 | Helped |
Surprising findings
- Global TV reward did not help and was set to zero, while the incremental TV reward was beneficial.
- Multi-scale frontier maps were crucial; removing them dropped lawn mowing coverage from 97.8% to 72.6% in ablation.
Limitations
- Assumes known pose up to noise; SLAM integration left to future work.
- Lawn mowing still lags offline BSA by about 35% at T90 and 51% at T99.
- Moving obstacles and multi-agent coverage are out of scope.
Experimental setup
Datasets: Explore-Bench maps randomly generated floor plan and obstacle maps
Benchmarks: Explore-Bench lawn mowing evaluation maps
Metrics: T90 time to reach 90% coverage T99 time to reach 99% coverage coverage percentage at fixed time inference time collision frequency
Baselines: Distance frontier RRT frontier Potential field frontier Active Neural SLAM Frontier RL (non-omnidirectional) Backtracking spiral algorithm TSP with A*
Builds on: soft actor-critic Active Neural SLAM Explore-Bench
Closest papers by meaning
- Large-Scale Multi-Robot Coverage Path Planning via Local Search (AAAI 2024 · similarity 0.85)
- Scalable Online Exploration via Coverability (ICML 2024 · similarity 0.85)
- State-Covering Trajectory Stitching for Diffusion Planners (NeurIPS 2025 · similarity 0.83)
- Learn to Follow: Decentralized Lifelong Multi-Agent Pathfinding via Planning and Learning (AAAI 2024 · similarity 0.83)
- Imagine Beyond ! Distributionally Robust Autoencoding for State Space Coverage in Online Reinforcement Learning (NeurIPS 2025 · similarity 0.83)
- Exploring the Edges of Latent State Clusters for Goal-Conditioned Reinforcement Learning (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.