AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

DePO: Demonstration-guided Policy Optimization for Molecular Optimization

Xuan Li, Zhanke Zhou, Zongze Li, Jiangchao Yao, Yu Rong, Lu Zhang, Bo Han

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DePO, a demonstration-guided policy optimization method for adapting LLMs to molecular optimization, where datasets contain only reference molecules and objectives are competing.

Problem

Adapting LLMs to molecular optimization is hard because its datasets provide only reference molecules without reasoning traces (ruling out SFT on reasoning traces), and its competing objectives hinder RLVR.

Approach

Uses reference molecules from the dataset as demonstrations to guide policy optimization, addressing the lack of reasoning traces and the difficulty of competing objectives under RLVR. Full method details are unavailable from the provided abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add reference guidance to GRPO (RePO)Qwen-2.5-3B-Instruct · TOMG-Bench AddComponentSR x Similarity: 0.005 -> 0.239Improved
add reference guidance to GRPOQwen-2.5-3B-Instruct · TOMG-Bench MRSR x Similarity: 0.188 -> 0.294Improved
add reference guidance to GRPOQwen-2.5-3B-Instruct · TOMG-Bench LogPSR x Similarity: 0.305 -> 0.297No clear effect
use RePO instead of GRPO initialized from answer-only SFTQwen-2.5-3B-Instruct · TOMG-Bench DelComponentSR x Similarity: 0.176 -> 0.140Hurt
initialize GRPO from answer-only SFTGRPO · molecular optimizationresponse length / successHurt
use RePO instead of SFT on multi-objective seen tasksQwen-2.5-3B-Instruct · MuMOInstruct BPQ seenSR x Similarity: 0.115 -> 0.139Improved
use RePO instead of GRPO under unseen instructionsQwen-2.5-3B-Instruct · MuMOInstruct BPQ unseenSR x Similarity: 0.107 -> 0.144Improved
use RePO instead of GRPO for multi-objective BDP seenQwen-2.5-3B-Instruct · MuMOInstruct BDP seenSR x Similarity: 0.118 -> 0.117No clear effect
apply RePO on Llama-3.1-8BLlama-3.1-8B-Instruct · TOMG-Bench LogPSR x Similarity: 0.219 -> 0.269Improved
apply RePO on larger backboneQwen-2.5-7B-Instruct · TOMG-Bench QEDSR x Similarity: 0.252 -> 0.213Hurt
mask gradients of answer supervision through reasoning tokens (ablation)RePO · TOMG-Bench property tasksSR x SimilarityImproved
use continuous instead of binary property reward (ablation)RePO · TOMG-Bench QEDSR x Similarity: 0.236 -> 0.203Hurt

Experimental setup

Builds on: RLVR SFT

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy