Research map / Reinforcement learning / Visual and representation RL
Revisiting OOD Generalization in Programmatic RL
Amirhossein Rajabpour, Kiarash Aghakasiri, Sandra Zilles, Levi Lelis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a cautious intrinsic reward (beta 0.5 instead of 1.0) for the neural policy (ablation) | DDPG policy in TORCS · TORCS OOD tracks | generalization to unseen tracks: CR 1608M (all crashed) -> 1:48 (0.76) on G-TRACK-2 | Improved |
| lower beta in the reward, which slows the car (ablation) | DDPG policy in TORCS · G-TRACK-1 training track | lap time: 54 -> 1:17 | Hurt |
| use cautious reward when training on AALBORG (ablation) | DDPG policy in TORCS · ALPINE-2 and RUUDSKOGEN | fraction of seeds generalizing: CR 1688M -> 3:13 (1.00); CR 3232M -> 2:46 (1.00) | Improved |
| augment observation with the agent's last action in a fully connected network (instead of LSTM or ConvNet) | PPO in partially observable KAREL · KAREL TOPOFF 100x100 | average return: 0.15 -> 1.00 (vs LSTM) | Improved |
| use a_{t-1} with partial observation instead of ConvNet with full observation | PPO in KAREL · KAREL STAIRCLIMBER 100x100 | average return: 0.00 -> 1.00 | Improved |
| use a_{t-1} instead of ConvNet on HARVESTER | PPO in KAREL · KAREL HARVESTER | average return: 0.90 -> 0.59 (small); 0.00 -> 0.04 (100x100) | Mixed |
| use an LSTM for partial observability instead of a_{t-1} | PPO in KAREL · KAREL STAIRCLIMBER small | average return: 1.00 -> 0.13 | Hurt |
| neural policy (DQN) vs programmatic PSM on a tight-gap test | PARKING · PARKING test | success rate: 0.16 (PSM) vs 0.18 (DQN) | No clear effect |
| compare generalization gap between train and test | PSM vs DQN in PARKING · PARKING | success rate drop: PSM 0.26 -> 0.16; DQN 0.86 -> 0.18 | Mixed |
Closest papers by meaning
- Generative OOD-regularized Model-based Policy Optimization (NeurIPS 2026 · similarity 0.88)
- On the Global Optimality of Policy Gradient Methods in General Utility Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- On Generalization Across Environments In Multi-Objective Reinforcement Learning (ICLR 2025 · similarity 0.87)
- Horizon Generalization in Reinforcement Learning (ICLR 2025 · similarity 0.87)
- Reclaiming the Source of Programmatic Policies: Programmatic versus Latent Spaces (ICLR 2024 · similarity 0.87)
- Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
