Research map / Optimization / Distributed and compressed
Embarrassingly Parallel GFlowNets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EP-GFlowNet, a divide-and-conquer method that trains local GFlowNets in parallel on client rewards and aggregates their policies on a server via a new aggregating balance condition to sample from the product of rewards in discrete spaces.
Problem
Sampling from a product of local discrete reward distributions (e.g., parallel or federated Bayesian posteriors over discrete objects) with few client-server communication rounds and without repeated access to all data; existing embarrassingly parallel MCMC methods only handle continuous variables.
Approach
Train local GFlowNets per client in parallel, then learn a global GFlowNet by enforcing an aggregating balance condition on their forward/backward policies, which is provably equivalent to sampling proportionally to the product of local rewards; it also introduces contrastive balance, a loss needing only forward and backward policies.
Key results
- On grid world, EP-GFlowNet's L1 error is 0.038 versus 0.027 for the centralized GFlowNet and 0.189 for PCVI, with identical top-800 reward to the centralized model.
- On phylogenetics, EP-GFlowNet reaches an L1 error of 0.088 against the target product distribution, comparable to the average client error of 0.083.
- On multisets and sequences, EP-GFlowNet is roughly 8x more accurate than PCVI in L1 distance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EP-GFlowNet instead of PCVI | Parallel reward aggregation · Grid World | L1 distance: 0.189 -> 0.038 | Helped |
| Use EP-GFlowNet instead of PCVI | Parallel reward aggregation · Multisets | L1 distance: 0.834 -> 0.130 | Helped |
| Use EP-GFlowNet instead of PCVI | Parallel reward aggregation · Sequences | L1 distance: 1.872 -> 0.005 | Helped |
| Use EP-GFlowNet instead of PCVI | Parallel reward aggregation · Sequences | Top-800 average log reward: -16.473 -> -1.535 | Helped |
| Use EP-GFlowNet instead of a centralized GFlowNet | Reward aggregation · Grid World | Top-800 average log reward: -6.355 -> -6.355 | No clear effect |
| Use EP-GFlowNet instead of a centralized GFlowNet | Reward aggregation · Multisets | Top-800 average log reward: 27.422 -> 27.422 | No clear effect |
| Use EP-GFlowNet instead of a centralized GFlowNet | Reward aggregation · Sequences | Top-800 average log reward: -1.535 -> -1.535 | No clear effect |
| Train local GFlowNets with contrastive balance instead of TB or DB | Centralized GFlowNet training · Multiset generation, phylogeny, and BNSL | L1 error convergence | Helped |
| Train with contrastive balance instead of DB | Centralized GFlowNet training · Grid World and sequences | L1 error convergence | No clear effect |
| Train with contrastive balance instead of TB | Multiset generation · Multisets | L1 error convergence | Helped |
Surprising findings
- PCVI frequently samples invalid elements outside the posterior support in phylogenetics, so it was not compared there.
- Minimizing the aggregating balance objective can still give a good approximation of the product of marginals even when one local GFlowNet is poorly trained, yet the result diverges from the target product distribution.
Limitations
- Guarantees hold only when local GFlowNets are perfectly trained; the error bound degenerates when a local model is poorly trained.
- Experiments are on small synthetic or moderate-size discrete tasks with uniform backward policies.
Experimental setup
Datasets: synthetic grid world synthetic multisets synthetic sequences phylogenetic nucleotide sequences (2500 sites) synthetic linear SEM data for federated BNSL
Benchmarks: grid world multisets sequences Bayesian phylogenetic inference federated Bayesian network structure learning
Metrics: L1 distance to target distribution top-800 average log reward training convergence (L1 vs. iterations)
Baselines: centralized GFlowNet PCVI trajectory balance detailed balance
Builds on: GFlowNet trajectory balance detailed balance parallel categorical VI (SDA-Bayes) DAG-GFlowNet
Closest papers by meaning
- Streaming Bayes GFlowNets (NeurIPS 2024 · similarity 0.88)
- On Divergence Measures for Training GFlowNets (NeurIPS 2024 · similarity 0.87)
- Local Search GFlowNets (ICLR 2024 · similarity 0.86)
- Order-Preserving GFlowNets (ICLR 2024 · similarity 0.85)
- Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets (ICLR 2025 · similarity 0.84)
- gfnx: Fast and Scalable Library for Generative Flow Networks in JAX (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.