Research map / Reinforcement learning / Offline RL

Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning

NeurIPS 2026 · Reinforcement learning › Offline RL · approach: Offline RL sample complexity theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoDADA, which clusters merged source/target next states with K-means, estimates per-cluster KL dynamics discrepancy via cluster-wise domain classifiers, filters and reweights source transitions, and adds a CVAE target-behavior regularizer on top of IQL for off-dynamics offline RL.

Problem

Off-dynamics offline RL: learning a target-domain policy from scarce target data plus abundant source data with different transition dynamics; existing methods handle mismatch globally or via costly pointwise filtering, missing localized similarities.

Approach

Localize the dynamics mismatch by clustering next states, estimate a cluster-level KL divergence with cluster-specific domain classifiers (density-ratio estimate), retain source transitions from low-KL clusters, reweight by pointwise KL in the critic loss, and regularize the policy toward a CVAE-fit target behavior policy.

Key results

• On MuJoCo gravity and friction shifts, LoDADA reaches a total score of 1207.05 vs 1007.10 for OTDF (reported 19.9% improvement) and 894.82 for IQL.
• On localized perturbation shifts, LoDADA achieves a total of 943.09 vs 729.44 for OTDF (reported 29.3% improvement).
• On AntMaze medium with diverse source data, LoDADA totals 416.4 vs 384.1 for IQL (reported 8.4% improvement over second-best).

Surprising findings

• Plain IQL trained on mixed data outperforms several dedicated off-dynamics methods (e.g., DARA, IGDF) on the localized perturbation benchmark in total score.

Limitations

• Sensitive to the regularization weight lambda, with best value varying across tasks.
• Results depend on the number of clusters K; too small or too large K degrades performance.
• Theoretical bound relies on approximations (e.g., P(z|y) approximated by P(z|s'_src/tar)) and assumed bounded KL.

Experimental setup

Datasets: ODRL benchmark datasets D4RL MuJoCo datasets D4RL AntMaze diverse and play datasets D4RL Adroit datasets

Benchmarks: ODRL MuJoCo gravity and friction shifts ODRL MuJoCo local perturbation shifts ODRL AntMaze medium ODRL Adroit pen and door

Metrics: normalized score total score across tasks

Baselines: BOSA IQL DARA IGDF OTDF

Builds on: Implicit Q-Learning ODRL benchmark OTDF BOSA DARA

Closest papers by meaning

• Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning (AAAI 2024 · similarity 0.93)
• ODRL: A Benchmark for Off-Dynamics Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
• Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learning (ICML 2024 · similarity 0.91)
• Cross-Domain Policy Adaptation by Capturing Representation Mismatch (ICML 2024 · similarity 0.90)
• A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learning (ICML 2025 · similarity 0.90)