Research map / Reinforcement learning / RL from feedback
Effects of Structural Reward Shaping on Biophysical Properties in RL-Trained Plasmid Generators
McClain Thiel, Angus G. Cunningham, Chris P Barnes
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the cassette arrangement bonus (promoter-CDS-terminator order) to the reward (ablation) | GRPO-trained PlasmidGPT · 8 prompts, T=0.95 | QC pass rate: 19.8% -> 66.9% | Improved |
| add GRPO structural-reward post-training to pretrained PlasmidGPT | PlasmidGPT · 8 prompts, 4,000 sequences | QC pass rate: 4.3% -> 71.6% | Improved |
| supervised fine-tuning on curated plasmids instead of RL | PlasmidGPT · 8 prompts, 4,000 sequences | QC pass rate: 4.3% -> 11.0% | Improved |
| rejection sampling more heavily from the base model (K=64) (ablation) | Base PlasmidGPT · Rejection sampling success | success rate: 4.3% -> 54.5% (K=1 to K=64), still far below RL 76.8% at K=1 | No clear effect |
| RL post-training reduces output diversity | PlasmidGPT · pooled 21-mer Jaccard distance | diversity: 0.960 -> 0.596 | Hurt |
| RL post-training on held-out plasmid continuation vs Base | PlasmidGPT · 29 held-out non-Addgene plasmids | mean log-probability: -8.99 -> -8.16 | Improved |
| RL instead of SFT on held-out continuation log-probability | PlasmidGPT · 29 held-out non-Addgene plasmids | mean log-probability: -7.35 -> -8.16 | Hurt |
| add structural reward post-training (SFT or RL) on MFE density | PlasmidGPT · Generated sequences | MFE density (kcal/mol/bp): -0.105 -> -0.149 (real reference -0.151) | Improved |
| use only a single reward component (length-only or CDS-only) (ablation) | GRPO reward · MFE density of generated sequences | MFE density (kcal/mol/bp): -0.149 (full) vs -0.126 (length-only), -0.103 (CDS-only) | Hurt |
| RL reward shaping on GC content | PlasmidGPT · Generated sequences | GC content: RL 0.524 vs real 0.510 | Mixed |
Closest papers by meaning
- Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models (NeurIPS 2024 · similarity 0.86)
- Iterative Foundation Model Fine-Tuning on Multiple Rewards (NeurIPS 2025 · similarity 0.84)
- Regulatory DNA Sequence Design with Reinforcement Learning (ICLR 2025 · similarity 0.84)
- Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design (ICML 2025 · similarity 0.84)
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.84)
- Improved Off-policy Reinforcement Learning in Biological Sequence Design (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
