Research map / Reinforcement learning / Visual and representation RL
MANDREL: Modular Reinforcement Learning Pipelines for Material Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents MANDREL, a modular Python framework (Gymnasium-based) that mixes action spaces, molecular featurisers, reward functions and RL algorithms for RL-driven molecular design, with a Dash demo interface.
Problem
Material and molecular discovery RL pipelines are fragmented and monolithic, making it hard to compare or reuse components across pipelines.
Approach
A modular RL toolkit where reward function, action space (environment) and featuriser are composed into a Gymnasium MDP, and any RL algorithm from SB3 or RLlib can be plugged in, covering SMILES, SELFIES, STONED-SELFIES and Zhou et al. action spaces.
Key results
- Recovers QED results of Zhou et al. (2019) and finds masked DQN outperforms vanilla DQN and masked PPO from SB3.
- SELFIES environment yields a stable agent with reduced reward variance but needs more steps to converge.
- STONED-SELFIES environment shows behaviour similar to Zhou et al. (2019).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the masked DQN variant instead of vanilla DQN | MANDREL QED pipeline · QED | performance | Helped |
| use the masked DQN variant instead of masked PPO | MANDREL QED pipeline · QED | performance | Helped |
| switch from the SMILES environment to the SELFIES environment | MANDREL agent · QED | reward variance | Helped |
| switch from the SMILES environment to the SELFIES environment | MANDREL agent · QED | steps to converge | Hurt |
| switch to the STONED-SELFIES environment | MANDREL agent · QED | behavior | No clear effect |
Limitations
- Demo paper with limited quantitative evaluation
- Only illustrative reward functions included
- Carbon capture rewards depend on surrogate models
Experimental setup
Benchmarks: QED optimisation PLogP optimisation
Metrics: reward over training curves
Baselines: vanilla DQN masked PPO
Builds on: Zhou et al. 2019 masked DQN SELFIES STONED-SELFIES Stable-Baselines3 Gymnasium
Closest papers by meaning
- Searching for High-Value Molecules Using Reinforcement Learning and Transformers (ICLR 2024 · similarity 0.89)
- MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures (NeurIPS 2025 · similarity 0.88)
- MADE: Benchmark Environments for Closed-Loop Materials Discovery (ICML 2026 · similarity 0.87)
- Uncertainty-Aware Multi-Objective Reinforcement Learning-Guided Diffusion Models for 3D De Novo Molecular Design (NeurIPS 2025 · similarity 0.86)
- Aligning Transformers with Continuous Feedback via Energy Rank Alignment (NeurIPS 2025 · similarity 0.85)
- Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.