Research map / Reinforcement learning / Offline RL
Causal Flow Q-Learning for Robust Offline Reinforcement Learning
Mingxuan Li, Junzhe Zhang, Elias Bareinboim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add confounding-robust worst-case Q target with discriminator and Q-ensemble minimum to FQL | FQL · OGBench visual tasks, offline (25 tasks) | normalized mean success: 1.0 -> 1.21 | Improved |
| add confounding-robust objective to FQL on cube-single | FQL · visual-cube-single-play (5 tasks) | success rate (%): 65 -> 81 | Improved |
| add confounding-robust objective to FQL on cube-double | FQL · visual-cube-double-play (5 tasks) | success rate (%): 6 -> 11 | Improved |
| add confounding-robust objective to FQL on puzzle-3x3 task5 | FQL · visual-puzzle-3x3-play task5 | success rate (%): 1 -> 19 | Improved |
| add confounding-robust objective to FQL on scene task5 | FQL · visual-scene-play task5 | success rate (%): 0 -> 0 | No clear effect |
| set discriminator loss coefficient too small (ablation) | Causal-FQL · visual-cube-double task1 (coefficient sweep {1, 5, 10, 15}) | success rate | Hurt |
| use more Q-network ensembles (2, 4, 6) (ablation) | Causal-FQL · visual-cube-double task1 | success rate | Hurt |
| use Causal-FQL for offline-to-online fine-tuning | FQL · OGBench visual tasks, offline-to-online | success rate: 8% improvement on average | Improved |
| switch back to vanilla FQL objective during online phase (cube tasks) (ablation) | Causal-FQL · visual-cube-single and visual-cube-double, offline-to-online | success rate | Improved |
| balanced sampling of online and offline buffers with matching objectives (puzzle-4x4) (ablation) | Causal-FQL · visual-puzzle-4x4, offline-to-online | success rate | Improved |
| use pixel observations instead of structured state observations | SOTA flow policy offline RL · OGBench, 5 tasks per category | normalized success rate: nearly 80% drop in some | Hurt |
Closest papers by meaning
- Flow Q-Learning (ICML 2025 · similarity 0.92)
- Flow Matching for Offline Reinforcement Learning with Discrete Actions (NeurIPS 2026 · similarity 0.90)
- Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data (NeurIPS 2025 · similarity 0.90)
- Confounding Robust Deep Reinforcement Learning: A Causal Approach (NeurIPS 2025 · similarity 0.89)
- Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding (ICLR 2024 · similarity 0.89)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
