Research map / Large language models / Reasoning and chain of thought
DePO: Demonstration-guided Policy Optimization for Molecular Optimization
Xuan Li, Zhanke Zhou, Zongze Li, Jiangchao Yao, Yu Rong, Lu Zhang, Bo Han
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DePO, a demonstration-guided policy optimization method for adapting LLMs to molecular optimization, where datasets contain only reference molecules and objectives are competing.
Problem
Adapting LLMs to molecular optimization is hard because its datasets provide only reference molecules without reasoning traces (ruling out SFT on reasoning traces), and its competing objectives hinder RLVR.
Approach
Uses reference molecules from the dataset as demonstrations to guide policy optimization, addressing the lack of reasoning traces and the difficulty of competing objectives under RLVR. Full method details are unavailable from the provided abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reference guidance to GRPO (RePO) | Qwen-2.5-3B-Instruct · TOMG-Bench AddComponent | SR x Similarity: 0.005 -> 0.239 | Improved |
| add reference guidance to GRPO | Qwen-2.5-3B-Instruct · TOMG-Bench MR | SR x Similarity: 0.188 -> 0.294 | Improved |
| add reference guidance to GRPO | Qwen-2.5-3B-Instruct · TOMG-Bench LogP | SR x Similarity: 0.305 -> 0.297 | No clear effect |
| use RePO instead of GRPO initialized from answer-only SFT | Qwen-2.5-3B-Instruct · TOMG-Bench DelComponent | SR x Similarity: 0.176 -> 0.140 | Hurt |
| initialize GRPO from answer-only SFT | GRPO · molecular optimization | response length / success | Hurt |
| use RePO instead of SFT on multi-objective seen tasks | Qwen-2.5-3B-Instruct · MuMOInstruct BPQ seen | SR x Similarity: 0.115 -> 0.139 | Improved |
| use RePO instead of GRPO under unseen instructions | Qwen-2.5-3B-Instruct · MuMOInstruct BPQ unseen | SR x Similarity: 0.107 -> 0.144 | Improved |
| use RePO instead of GRPO for multi-objective BDP seen | Qwen-2.5-3B-Instruct · MuMOInstruct BDP seen | SR x Similarity: 0.118 -> 0.117 | No clear effect |
| apply RePO on Llama-3.1-8B | Llama-3.1-8B-Instruct · TOMG-Bench LogP | SR x Similarity: 0.219 -> 0.269 | Improved |
| apply RePO on larger backbone | Qwen-2.5-7B-Instruct · TOMG-Bench QED | SR x Similarity: 0.252 -> 0.213 | Hurt |
| mask gradients of answer supervision through reasoning tokens (ablation) | RePO · TOMG-Bench property tasks | SR x Similarity | Improved |
| use continuous instead of binary property reward (ablation) | RePO · TOMG-Bench QED | SR x Similarity: 0.236 -> 0.203 | Hurt |
Experimental setup
Builds on: RLVR SFT
Closest papers by meaning
- Aligning Transformers with Continuous Feedback via Energy Rank Alignment (NeurIPS 2025 · similarity 0.87)
- Searching for High-Value Molecules Using Reinforcement Learning and Transformers (ICLR 2024 · similarity 0.87)
- DEPO: Dual-Efficiency Preference Optimization for LLM Agents (AAAI 2026 · similarity 0.86)
- LICO: Large Language Models for In-Context Molecular Optimization (ICLR 2025 · similarity 0.86)
- Amortized Molecular Optimization via Group Relative Policy Optimization (NeurIPS 2026 · similarity 0.85)
- On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimization (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
