Research map / Reinforcement learning / Offline RL
Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DynAMO, a distribution-matching KL penalty plus WGAN source-critic constraint that can be added to any offline MBO optimizer to boost candidate diversity while preserving quality.
Problem
Offline model-based optimization typically proposes low-diversity candidate designs (reward hacking / mode collapse), and surrogate models overestimate out-of-distribution designs.
Approach
Reformulates MBO as a KL-regularized objective that matches the tau-weighted offline dataset distribution, plus a WGAN source-critic constraint limiting surrogate exploitation; derives a closed-form concave lower bound on the Lagrange dual so the multiplier can be solved exactly, making the method optimizer-agnostic.
Key results
- DynAMO-BO-qUCB reaches an optimality gap of 74.2 on pairwise diversity... specifically DynAMO-BO-qEI reaches a diversity optimality gap of 74.2 versus at most 51.4 for baselines within that optimizer class.
- DynAMO achieves an average rank as low as 1.2 across six tasks for Grad., Adam, and CoSyNE backbones.
- DynAMO is best on Best@128 optimality gap for 5 of 6 backbone optimizers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add KL distribution matching and adversarial source-critic constraint | Gradient Ascent · Six-task average, Pairwise Diversity | Optimality Gap: 27.8 vs -6.9 | Helped |
| Add KL distribution matching and adversarial source-critic constraint | CMA-ES · Six-task average, Pairwise Diversity | Optimality Gap: 55.2 vs 16.8 | Helped |
| Add KL distribution matching and adversarial source-critic constraint | BO-qEI · Six-task average, Pairwise Diversity | Optimality Gap: 74.2 vs 51.4 | Helped |
| Add KL distribution matching and adversarial source-critic constraint | DynAMO with CoSyNE · Six-task average, Pairwise Diversity | Optimality Gap: -20.7 | Mixed |
Surprising findings
- DynAMO does not match baselines on median (Median@128) quality, which the authors acknowledge as a limitation.
Limitations
- Median@128 quality is worse than baselines, so no method dominates all metrics.
- Evaluated only on well-studied Design-Bench-style tasks; noisy, sparse, or high-dimensional real-world settings untested.
- One-shot batched evaluation only; not extended to active learning.
Experimental setup
Datasets: TFBind8 UTR ChEMBL Superconductor D'Kitty Molecule
Benchmarks: TFBind8 UTR ChEMBL Superconductor D'Kitty Molecule
Metrics: Best@128 Pairwise Diversity Rank Optimality Gap Median@128
Baselines: COMs RoMA ROMO GAMBO Baseline (unmodified MBO)
Builds on: GAMBO Wasserstein GAN Design-Bench CMA-ES
Closest papers by meaning
- Preference-Guided Diffusion for Multi-Objective Offline Optimization (NeurIPS 2025 · similarity 0.86)
- ROOT: Rethinking Offline Optimization as Distributional Translation via Probabilistic Bridge (NeurIPS 2025 · similarity 0.86)
- Offline Model-Based Optimization by Learning to Rank (ICLR 2025 · similarity 0.85)
- Learning Surrogates for Offline Black-Box Optimization via Gradient Matching (ICML 2024 · similarity 0.85)
- ParetoFlow: Guided Flows in Multi-Objective Optimization (ICLR 2025 · similarity 0.85)
- Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.