Research map / Reinforcement learning / RL from feedback
AReaL-DTA: Dynamic Tree Attention for Efficient Reinforcement Learning of Large Language Models
Jiarui Zhang, Yuchen Yang, Ran Yan, Zhiyu Mei, Liyuan Zhang, Daifeng Li, Wei Fu, Jiaxuan Gao, Shusheng Xu, Yi Wu, Binhang Yuan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add tree traversal of prefix tree in backward pass (ablation) | AReaL-DTA · tau2-bench single-GPU throughput | speedup vs Dense+CKPT: 7.53x | Improved |
| add DFS traversal order (ablation) | AReaL-DTA Tree · tau2-bench single-GPU throughput | speedup: 7.53x -> 7.74x | Improved |
| add larger chunk block size (4096) (ablation) | AReaL-DTA Tree+DFS · tau2-bench single-GPU throughput | speedup: 7.74x -> 8.31x | Improved |
| add load-balanced prefix-tree partitioning (ablation) | AReaL-DTA · tau2-bench data-parallel training | wall-clock performance: 11.93% | Improved |
| use AReaL-DTA instead of Sparse tree-mask with checkpointing | training · tau2-bench | throughput: 1.52x-1.70x | Improved |
| use AReaL-DTA instead of dense training | Qwen3-1.7B · tau2-bench | peak memory: over 50% reduction | Improved |
| use AReaL-DTA instead of AReaL for end-to-end RL | Qwen3-8B · tau2-bench | end-to-end throughput: 2.28x | Improved |
| use AReaL-DTA instead of AReaL for end-to-end RL | Qwen3-1.7B · tau2-bench | end-to-end throughput: 1.28x | Improved |
| use AReaL-DTA instead of AReaL for end-to-end RL | Qwen3-1.7B/8B · tau2-bench | reward vs training step | No clear effect |
| shift GPU split to 5/3 rollout/training with AReaL-DTA (ablation) | AReaL-DTA · tau2-bench Qwen3-8B | throughput K tokens/s: 27.5 -> 51.4 | Improved |
| apply AReaL-DTA on low-compression GSM8K workload | Qwen3-1.7B · GSM8K (C=1.09) | relative throughput: 1.00x -> 0.62x | Hurt |
| apply AReaL-DTA on AIME workload | Qwen3-1.7B · AIME (C=1.008) | relative throughput: 1.00x -> 1.19x | Improved |
Closest papers by meaning
- AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning (NeurIPS 2025 · similarity 0.92)
- DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training (NeurIPS 2026 · similarity 0.89)
- On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.88)
- d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- ROLLVERIFY: BRIDGING EFFICIENCY AND ACCURACY IN LONG-TAIL ROLLOUT REINFORCEMENT LEARNING (NeurIPS 2026 · similarity 0.88)
- Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
