Research map / Reinforcement learning / Visual and representation RL
Pre-Training and Fine-Tuning Generative Flow Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes reward-free unsupervised pre-training of GFlowNets via an outcome-conditioned GFlowNet (OC-GFN), plus an amortized predictor for efficient fine-tuning to new reward functions.
Problem
GFlowNets are trained from scratch for each new extrinsic reward function; there is no efficient way to pre-train them without rewards and adapt them to downstream tasks.
Approach
Pre-train an outcome-conditioned GFlowNet that learns to reach any target outcome (reward = 1 if reached), using GAFN-driven outcome generation, contrastive training and outcome teleportation; convert it to a new reward's policy by marginalizing over outcomes, approximated by a learned amortized numerator network.
Key results
- OC-GFN reaches near 100% outcome-reaching success rate at end of pre-training in GridWorld, TFBind8 and RNA tasks.
- On TFBind8 across 30 downstream tasks, OC-GFN has mean rank 1.27 for number of modes (GFN from scratch 1.97, MCMC 2.77, DQN 4.00).
- OC-GFN achieves much higher mode diversity than baselines on bit sequence, RNA and AMP generation tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pre-train OC-GFN and adapt with the amortized approach instead of training a GFN from scratch | GFlowNet · TF Bind 8 downstream tasks (30 tasks) | mean rank, number of modes: 1.97 -> 1.27 | Helped |
| Pre-train OC-GFN and adapt with the amortized approach instead of training a GFN from scratch | GFlowNet · TF Bind 8 downstream tasks (30 tasks) | mean rank, Top-K reward: 2.00 -> 1.20 | Helped |
| Add outcome teleportation to OC-GFN pre-training (ablation) | OC-GFN · GridWorld, especially larger maps | sample efficiency | Helped |
| Add contrastive training to OC-GFN pre-training (ablation) | OC-GFN without outcome teleportation · GridWorld, larger outcome spaces | outcome-reaching success | Helped |
| Add outcome teleportation to OC-GFN pre-training (ablation) | OC-GFN with contrastive training · Bit sequence generation, large outcome spaces | outcome-reaching success | Helped |
| Use OC-GFN with amortized adaptation instead of training a GFN from scratch | GFlowNet · Bit sequence generation downstream tasks | number of modes discovered | Helped |
| Use OC-GFN with amortized adaptation instead of baselines | GFlowNet · TF Bind 8 downstream tasks | number of modes and Top-K score | Helped |
| Use OC-GFN pre-training and transfer instead of baselines | GFlowNet · RNA generation, four downstream tasks | number of modes | Helped |
| Use OC-GFN pre-training and transfer instead of baselines | GFlowNet · Antimicrobial peptide generation, length 50 | number of modes | Helped |
Surprising findings
- Without outcome teleportation and contrastive training, OC-GFN fails to learn in large outcome spaces, as the agent rarely reaches targets.
Limitations
- Outcome space is restricted to discrete outcomes; continuous outcomes left to future work.
- Sparse binary outcome reward makes training hard in long-horizon, high-dimensional tasks, addressed only by contrastive training and teleportation.
- Code will be released only upon publication; baseline implementations rely on the open-source GFN codebase.
Experimental setup
Datasets: GridWorld bit sequence generation TFBind8 (30 downstream tasks) RNA binding (ViennaRNA) antimicrobial peptides
Benchmarks: GridWorld bit sequence generation TFBind8 RNA generation (ViennaRNA) antimicrobial peptide generation
Metrics: success rate number of modes discovered top-K reward mean rank
Baselines: GFN from scratch Metropolis-Hastings MCMC DQN trajectory balance
Builds on: GFlowNets generative augmented flow networks detailed balance trajectory balance goal-conditioned reinforcement learning
Closest papers by meaning
- Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets (ICLR 2025 · similarity 0.93)
- Flow Factorization for Efficient Generative Flow Networks (AAAI 2025 · similarity 0.92)
- Pessimistic Backward Policy for GFlowNets (NeurIPS 2024 · similarity 0.92)
- Order-Preserving GFlowNets (ICLR 2024 · similarity 0.92)
- GFlowNet Training by Policy Gradients (ICML 2024 · similarity 0.92)
- Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximization (ICLR 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.