Research map / Large language models / Agents, code and math

Entropy-Reinforced Planning with Large Language Models for Drug Discovery

ICML 2024 · Large language models › Agents, code and math · approach: LLMs for scientific discovery · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ERP, an MCTS-based planner with an e-step forward entropy bonus and Top-PK expansion that guides a GPT-2 SMILES Transformer decoder to generate drug-like molecules with better multi-property rewards, and also tests it on APPS code generation.

Problem

LLM decoding (beam search, sampling) for molecule generation yields many invalid molecules and suboptimal candidates for multi-property drug discovery, and prior MCTS-guided decoding (PG-TD) over-exploits high-probability tokens and wastes search on unpromising molecules.

Approach

Augments the P-UCT selection bonus with the Transformer's next-token probability multiplied by the average entropy of the policy over the next e steps, so the planner explores uncertain subtrees; expansion uses Top-PK filtering and evaluation uses beam search completions scored by a multi-critic normalized reward.

Key results

• On 3CLPro, ERP achieves best normalized reward 3.24 (+13.71% over sampling) versus PG-TD 3.20 (+12.50%).
• On RTCB, ERP achieves avg valid normalized reward 2.61 (+5.61% over sampling), with PG-TD at 2.53 (+2.26%).
• On APPS introductory with GPT-2, ERP improves 1@20 pass rate to 25.204% versus PG-TD 23.540%.

Measured effects reported in the paper

Surprising findings

• PG-TD's performance dropped when rollouts increased from 256 to 512, while ERP improved monotonically.
• Biased-model sampling performed significantly worse than pure MCTS UCT, indicating biased priors steer generation into low-reward regions.

Limitations

• Docking scores come from a surrogate BERT-like model rather than physical docking.
• Code generation experiments used reduced rollouts (64) due to limited compute, and differences are small (often within a few tenths of a percent).
• Reported improvements over sampling are modest on some metrics; ERP does not win every metric (e.g., unique valid molecules on 3CLPro vs UCT).

Experimental setup

Datasets: ZINC ZINC20 Cancer and COVID docking dataset (3CLPro, RTCB) APPS CodeContests

Benchmarks: 3CLPro (PDB 7BQY) molecule generation RTCB (PDB 4DWQ) molecule generation APPS Intro/Inter/Comp CodeContests

Metrics: best normalized reward average valid normalized reward average top 10% normalized reward unique valid molecules docking score druglikeness synthesizability solubility pass rate strict accuracy

Baselines: Beam Search Sampling UCT PG-TD

Builds on: PG-TD UCT P-UCT (value-guided MCTS) beam search

Closest papers by meaning

• Efficient Evolutionary Search Over Chemical Space with Large Language Models (ICLR 2025 · similarity 0.87)
• Aligning Transformers with Continuous Feedback via Energy Rank Alignment (NeurIPS 2025 · similarity 0.86)
• Searching for High-Value Molecules Using Reinforcement Learning and Transformers (ICLR 2024 · similarity 0.85)
• LLM-Augmented Chemical Synthesis and Design Decision Programs (ICML 2025 · similarity 0.85)
• Generating Novel Leads for Drug Discovery Using LLMs with Logical Feedback (AAAI 2024 · similarity 0.85)
• GenMol: A Drug Discovery Generalist with Discrete Diffusion (ICML 2025 · similarity 0.85)