Research map / Reinforcement learning / Offline RL
MGDA: Model-based Goal Data Augmentation for Offline Goal-conditioned Weighted Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MGDA, a model-based goal data augmentation method for GCWSL that uses a locally Lipschitz-regularized dynamics model to select reachable augmented goals, improving trajectory stitching on maze tasks.
Problem
Offline goal-conditioned weighted supervised learning (GCWSL) lacks trajectory stitching ability, and existing goal data augmentation methods (SGDA, TGDA) often pick inappropriate or unreachable augmented goals.
Approach
Establish three principles for goal data augmentation (goal diversity, action optimality, goal reachability) and implement them via a learned dynamics model trained with a local Lipschitz continuity regularization, which is used to find nearby states that reach the original goal and then sample later-trajectory goals as augmentations.
Key results
- MGDA improves all four GCWSL methods (WGCSL, GoFar, DWSL, SMORE) across state-based maze tasks, with gains most visible on Medium and Large.
- Table of DWSL with MGDA on 10^8 transitions reaches 91.2% Umaze, 63.4% Medium, 56.1% Large, versus 63.7%, 28.3%, 21.3% for DWSL alone on the same size dataset.
- MGDA outperforms SGDA and TGDA across tasks; TGDA is strong on Umaze but weaker on Medium and Large.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MGDA to DWSL | DWSL · Umaze | success rate (%): 50.2$\pm$3.9 -> 72.4$\pm$3.3; 50.6$\pm$8.0 -> 80.7$\pm$2.7; 55.5$\pm$4.5 -> 87.6$\pm$8.6; 63.7 $\pm$2.9 -> 91.2$\pm$1.8 | Helped |
| add MGDA to DWSL | DWSL · Medium | success rate (%): 21.3$\pm$2.3 -> 52.6$\pm$9.2; 22.3$\pm$2.3 -> 56.8$\pm$5.3; 24.1$\pm$1.5 -> 58.2$\pm$6.1; 28.3$\pm$5.8 -> 63.4$\pm$2.6 | Helped |
| add MGDA to DWSL | DWSL · Large | success rate (%): 11.3$\pm$4.9 -> 36.7$\pm$4.5; 16.3$\pm$6.7 -> 47.5$\pm$6.1; 15.8$\pm$8.9 -> 52.5$\pm$1.8; 21.3 $\pm$7.3 -> 56.1$\pm$3.9 | Helped |
| replace the local-Lipschitz modified MSE dynamics loss with standard MSE (ablation) | MGDA · point-maze goal-reaching tasks | success rate | Hurt |
| increase the DWSL dataset size instead of adding MGDA (ablation) | DWSL · point mazes | success rate | No clear effect |
Surprising findings
- Simply increasing offline dataset size does not improve DWSL generalization, suggesting more data does not expose more trajectory diversity.
- In vision-based Medium and Large tasks, MGDA gains were inconsistent, sometimes minimal.
Limitations
- Effectiveness of MGDA may depend on the number of layers in the neural network.
- MGDA did not consistently outperform existing augmentation methods on certain offline datasets, especially vision-based Medium and Large tasks.
Experimental setup
Datasets: goal-conditioned point maze datasets (Umaze, Medium, Large), state and vision-based
Benchmarks: goal-conditioned point maze (Umaze, Medium, Large) vision-based point maze
Metrics: mean success rate 95% bootstrap confidence intervals
Baselines: WGCSL GoFar DWSL SMORE SGDA TGDA
Builds on: GCWSL DWSL TGDA SGDA
Closest papers by meaning
- How to Solve Contextual Goal-Oriented Problems with Offline Datasets? (NeurIPS 2024 · similarity 0.85)
- Goal-Conditioned Supervised Learning for Multi-Objective Recommendation (NeurIPS 2026 · similarity 0.80)
- Score Models for Offline Goal-Conditioned Reinforcement Learning (ICLR 2024 · similarity 0.80)
- Conservative Offline Goal-Conditioned Implicit V-Learning (ICML 2025 · similarity 0.80)
- GOALNET: Interleaving Neural Goal Predicate Inference with Classical Planning for Generalization in Robot Instruction Following (AAAI 2024 · similarity 0.79)
- Stitching Sub-trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL (AAAI 2024 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.