Research map / Reinforcement learning / Visual and representation RL

Uncertainty-Aware Multi-Objective Reinforcement Learning-Guided Diffusion Models for 3D De Novo Molecular Design

NeurIPS 2025 · Reinforcement learning › Visual and representation RL · approach: RL with tree search for discovery · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an uncertainty-aware RL (PPO-style) framework that fine-tunes 3D molecular diffusion models (EDM and variants) using Chemprop surrogate-predicted property-satisfaction probabilities as rewards, evaluated on QM9, ZINC15, and PubChem with MD and ADMET analysis against EGFR inhibitors.

Problem

Controlling multiple property objectives (QED, SAS, docking binding affinity) when generating de novo 3D molecules with diffusion models, where properties are black-box and non-differentiable.

Approach

Use surrogate models with predictive uncertainty to compute per-property probabilities of meeting thresholds, multiply them into a joint success reward, add validity/uniqueness/novelty bonuses, a Tanimoto diversity penalty, and dynamic thresholds, and optimize the diffusion model with a PPO-style clipped policy gradient.

Key results

• On QM9, the method reaches 98.17% validity and 28.33% top-molecule ratio, versus 88.55% and 25.17% for the vanilla model without RL.
• On ZINC15, validity reaches 99.02% and top-molecule ratio 33.40%, compared with 30.05% validity and 8.02% top ratio for the W/O RL baseline.
• Surrogate models reach R² of 0.95–0.99 for QED and SAS and 0.86–0.88 for binding affinity, with AUCE 0.02–0.10.

Measured effects reported in the paper

Surprising findings

• On PubChem, validity remains low at 16.23%, which the authors attribute to backbone scalability for large molecules.
• Replacing the dynamic cutoff with a static cutoff gives higher validity (95.73%) but lower top-molecule ratio (24.88%) in ablation, indicating trade-offs.

Limitations

• Performance on PubChem with large, heavy-atom molecules is low (validity 16.23%), attributed to backbone scalability.
• Uniqueness slightly decreases relative to some baselines, acknowledged as a trade-off.
• Binding affinity is predicted by AutoDock Vina docking, which is a proxy for true binding.

Experimental setup

Datasets: QM9 ZINC15 PubChem EGFR crystal structure 6VHN

Benchmarks: QM9 ZINC15 PubChem

Metrics: validity uniqueness novelty VUN atom stability molecule stability top molecule percentage R2 AUCE

Baselines: W/O RL SFT-PG DDPO-SF DDPO-IS DPOK WS POO MMM LSDW

Builds on: EDM PPO DDPO Chemprop

Closest papers by meaning

• Navigating the Design Space of Equivariant Diffusion-Based Generative Models for De Novo 3D Molecule Generation (ICLR 2024 · similarity 0.89)
• Controllable 3D Molecular Generation for Structure-Based Drug Design Through Bayesian Flow Networks and Gradient Integration (NeurIPS 2025 · similarity 0.89)
• Multi-Objective Molecular Design Through Learning Latent Pareto Set (AAAI 2025 · similarity 0.88)
• DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization (ICLR 2024 · similarity 0.87)
• Multi-domain Distribution Learning for De Novo Drug Design (ICLR 2025 · similarity 0.87)
• BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning (AAAI 2025 · similarity 0.87)