Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Embodied Visual Active Learning for Semantic Segmentation
David Nilsson, Aleksis Pirinen, Erik Gärtner, Cristian Sminchisescu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a jointly learned exploration and annotation policy instead of comparable pre-specified policies | RL-agent · Matterport3D test scenes, 256 actions | mIoU: 0.385 -> 0.394 | Improved |
| Use a jointly learned exploration and annotation policy instead of comparable pre-specified policies | RL-agent · Matterport3D test scenes, 100-annotation budget | mIoU: 0.485 -> 0.507 | Improved |
| Add Collect actions and propagated annotations (ablation) | RL-agent · Matterport3D validation scenes | mIoU: 0.415 -> 0.427 | Improved |
| Learn annotation actions jointly with exploration instead of using a heuristic annotation strategy (ablation) | RL-agent · Matterport3D validation scenes | mIoU: 0.411 -> 0.427 | Improved |
| Add an explicit exploration reward (ablation) | RL-agent · Matterport3D validation scenes | mIoU: 0.401 -> 0.427 | Improved |
| Include visual features φ_img in the policy (ablation) | RL-agent · Matterport3D validation scenes | mIoU: 0.378 -> 0.427 | Improved |
| Include ResNet features φ_res in the policy (ablation) | RL-agent · Matterport3D validation scenes | mIoU: 0.375 -> 0.427 | Improved |
| Use learnt perception actions rather than threshold-triggered actions (ablation) | Space filler · Matterport3D validation scenes, space filler path | mIoU: 0.472 -> 0.454 | Hurt |
| Use randomized perception actions rather than threshold-triggered actions (ablation) | Space filler · Matterport3D validation scenes, space filler path | mIoU: 0.472 -> 0.446 | Hurt |
| Use learnt perception actions rather than threshold-triggered actions (ablation) | Bounce · Matterport3D validation scenes, bounce path | mIoU: 0.388 -> 0.375 | Hurt |
| Use randomized perception actions rather than threshold-triggered actions (ablation) | Bounce · Matterport3D validation scenes, bounce path | mIoU: 0.388 -> 0.379 | Hurt |
| Pre-train the segmentation network on training-house views before embodied RL (ablation) | RL-agent · Matterport3D validation scenes | mIoU: 0.427 -> 0.461 | Improved |
Closest papers by meaning
- EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI (CVPR 2024 · similarity 0.89)
- ACTIVE-o3 : Empowering MLLMs with Active Perception via Pure Reinforcement Learning (ICML 2026 · similarity 0.88)
- Selective Visual Representations Improve Convergence and Generalization for Embodied AI (ICLR 2024 · similarity 0.87)
- Online Segment Any 3D Thing as Instance Tracking (NeurIPS 2025 · similarity 0.87)
- I-Perceive: A Foundation Model for Vision-Language Active Perception (NeurIPS 2026 · similarity 0.87)
- SPA: 3D Spatial-Awareness Enables Effective Embodied Representation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
