Research map / Federated learning and privacy / Personalized and heterogeneous FL
Expected Returns and Policy Inconsistency-Aware Offline Federated Deep Reinforcement Learning
Meng XU, Zhongying Chen, Weiwei Fu, Yan Li, Shuguang Wang, Jianping Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add policy-inconsistency and Q-value aggregation plus decay to existing offline FDRL methods | FDQL · D4RL HalfCheetah | final D4RL score: 42.78 -> 48.55 | Improved |
| add the framework to FEDORA | FEDORA · D4RL Walker2d | final D4RL score: 67.99 -> 109.29 | Improved |
| add the framework to CSC | CSC · D4RL Walker2d | final D4RL score: 108.56 -> 65.12 | Hurt |
| add the framework to FOVA | FOVA · D4RL HalfCheetah | final D4RL score: 46.99 -> 42.35 | Hurt |
| add the framework to FDRLDR | FDRLDR · D4RL Hopper | final D4RL score: 44.44 -> 43.11 | Hurt |
| weight clients by both Q-value and policy inconsistency instead of policy inconsistency only (GAPI) (ablation) | Ours+FDRLDR/FORL · D4RL HalfCheetah | final D4RL score: 50.27 -> 51.74 (FORL) | Improved |
| weight clients by both criteria instead of Q-value only (GAQ) (ablation) | Ours+FORL · D4RL HalfCheetah | final D4RL score: 43.4 -> 51.74 | Improved |
| add decay factor on global model influence (zeta=0.99) versus none (ablation) | Ours+FORL · D4RL HalfCheetah | final D4RL score: 48.02 -> 51.74 | Improved |
| use stronger decay zeta 0.8 or 0.9 instead of 0.99 (ablation) | Ours+FORL · D4RL HalfCheetah | final D4RL score: 51.74 -> 46.84 (0.8), 44.34 (0.9) | Hurt |
| use JSD instead of squared difference for policy inconsistency (ablation) | Ours+FORL · D4RL HalfCheetah | final D4RL score: 49.2 -> 51.74 | Improved |
| use JSD instead of KLD for policy inconsistency (ablation) | Ours+FORL · D4RL HalfCheetah | final D4RL score: 49.76 -> 51.74 | Improved |
| use multiplicative importance (scaled Q times 1/Dis) instead of subtractive (ablation) | Ours+FORL · D4RL HalfCheetah | final D4RL score: 51.74 -> 2.66 | Hurt |
Closest papers by meaning
- Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices (ICML 2024 · similarity 0.89)
- Federated Ensemble-Directed Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning (ICLR 2024 · similarity 0.86)
- Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning (AAAI 2024 · similarity 0.86)
- ODRL: A Benchmark for Off-Dynamics Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
