Research map / Reinforcement learning / Offline RL
$\text{DT}^\text{2}$: Decision-Targeted Digital Twins
Harry Amad, Mihaela van der Schaar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with decision-targeted ranking loss instead of MSE | bottlenecked MLP digital twin · Limited neural capacity toy | ranking success rate: 20% -> 100% | Improved |
| train with decision-targeted ranking loss instead of MSE | MLP digital twin with partial observability · Partially observed toy | ranking accuracy: 65% -> 100% | Improved |
| add ranking loss to NLL simulation loss | expressive DT architectures · Six control environments | average regret: 4.08 -> 1.88 | Improved |
| add ranking loss to NLL simulation loss | expressive DT architectures · Six control environments | average Spearman: 0.45 -> 0.66 | Improved |
| add ranking loss to NLL simulation loss | expressive DT architectures · Six control environments | test transition MSE: 1.21 -> 1.41 | Hurt |
| add ranking loss to NLL simulation loss | ResNet DT · Cancer | regret: 59.56 -> 26.96 | Improved |
| add ranking loss to NLL simulation loss | ResNet DT · Cancer | Spearman: 0.10 -> 0.64 | Improved |
| add ranking loss to NLL simulation loss | ResNet DT · Cancer (unseen policies) | regret: 95.88 -> 49.27 | Improved |
| add ranking loss to NLL simulation loss | ResNet DT · Cancer (unseen policies) | Spearman: 0.40 -> 0.50 | Improved |
| use DT2 instead of FQE ranking directly | DT2 · Six control envs | regret, Spearman: Ant regret 0 vs 1.64; Walker regret 1.18 vs 0.21 | Mixed |
| compare MBRL pessimism methods MOReL, MOPO, ROMI with NLL DTs | offline MBRL baselines · Six control envs | regret, Spearman | No clear effect |
| increase ranking weight lambda from 0 up to 0.75 (ablation) | DT2 ResNet · Six control envs | Spearman and test MSE | Mixed |
Closest papers by meaning
- Automatically Learning Hybrid Digital Twins of Dynamical Systems (NeurIPS 2024 · similarity 0.87)
- Diffusion Policies Creating a Trust Region for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.86)
- Policy Filtration for RLHF to Mitigate Noise in Reward Models (ICML 2025 · similarity 0.85)
- Reflective Prompted Policy Optimization: Trajectory-Grounded Revision and Salience Bias (NeurIPS 2026 · similarity 0.85)
- Continuously Updating Digital Twins using Large Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
