Research map / Reinforcement learning / RL from feedback
ATPO: ADAPTIVE TREE POLICY OPTIMIZATION FOR MULTI-TURN MEDICAL DIALOGUE
Ruike Cao, Shaojie Bai, Fugen Yao, Liang Dong, Jian Xu, Li Xiao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add uncertainty-based adaptive pruning/expansion (ATPO U1+U2) over fixed binary TreePO | Qwen3-8B · MedQA | accuracy (%): 61.81 -> 64.07 | Improved |
| add uncertainty-based adaptive pruning/expansion (ATPO U1+U2) over TreePO | Qwen3-4B · MedQA | accuracy (%): 53.74 -> 55.47 | Improved |
| add uncertainty-based adaptive pruning/expansion (ATPO U1+U2) over TreePO | Qwen3-1.7B · MedQA | accuracy (%): 42.05 -> 42.87 | Improved |
| add second uncertainty metric U2 (variance) to U1 (ablation) | ATPO Qwen3-8B · MedMCQA | accuracy (%): 53.22 -> 53.66 | Mixed |
| add second uncertainty metric U2 (variance) to U1 (ablation) | ATPO Qwen3-1.7B · MedicalExam | accuracy (%): 45.73 -> 43.20 | Hurt |
| use tree search credit assignment (TreePO) instead of GRPO trajectory-level advantage | Qwen3-8B · MedQA | accuracy (%): 57.92 -> 61.81 | Improved |
| use hierarchical MDP instead of token-level MDP for PPO | PPO Qwen3-8B · MedQA | accuracy (%): 57.38 -> 57.81 | No clear effect |
| add interactive MEDIQ prompting over direct single-turn prompt | Qwen3-8B zero-shot · MedQA | accuracy (%): 45.22 -> 46.03 on MedQA; MedMCQA 46.16 -> 41.60 | Hurt |
| distill from GPT-4o or Gemini-2.5-Pro dialogues in SFT | Qwen3-8B · MedQA | accuracy (%): 53.25 and 51.51 vs SFT 53.75 | No clear effect |
| add RL (ATPO) on top of SFT | Qwen3-8B · MedQA | accuracy (%): 53.75 -> 64.07 | Improved |
| down-weight policy advantage by node visit count (vs no down-weighting, EXP1) (ablation) | ATPO · training stability | stability | Improved |
| also down-weight value loss by visit count (EXP2) (ablation) | ATPO · training stability | stability | Hurt |
Closest papers by meaning
- ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL (ICML 2024 · similarity 0.86)
- Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF (ICLR 2025 · similarity 0.86)
- Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (ICLR 2025 · similarity 0.86)
- Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-World Multi-Turn Dialogue (AAAI 2024 · similarity 0.86)
- Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training (ICLR 2025 · similarity 0.85)
- StateTree: Enhancing Long-term Dialogue Reasoning via Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
