Research map / Probabilistic and scientific modeling / Variational and generative sampling
Revisiting Non-Acyclic GFlowNets in Discrete Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits non-acyclic GFlowNets in discrete environments with a simpler theory built on backward policies, clarifies flows, state-flow regularization, and the GFlowNet–entropy-regularized RL connection, and tests loss stability on hypergrids and permutations.
Problem
GFlowNet theory and training assume acyclic environments; in non-acyclic (cyclic) graphs expected trajectory length can blow up, and existing theory relies on heavy measure-theoretic machinery.
Approach
Builds a discrete-state, backward-policy-centered theory of non-acyclic GFlowNets: shows fixed backward policies make losses stable, that expected trajectory length equals normalized total state flow (so minimizing length = minimizing total flow), proposes state flow regularization to DB, and generalizes the GFlowNet–soft RL equivalence to cycles.
Key results
- On the 20^4 hypergrid, log-flow-scale DB and SDB with state flow regularization learn the correct reward distribution, while flow-scale losses are biased.
- On permutations with n=8, DB with log-flow scale and lambda=1e-5 reaches C(k) L1 error 0.005 and reward error 0.001 but with E[n_tau] about 4.31, showing a tradeoff controlled by lambda.
- Without regularization, mean trajectory length under log-flow-scale losses tends to infinity on the larger hypergrid.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ΔF-scale losses without state flow regularization (ablation) | Trainable-backward-policy GFlowNets · Permutation n=20 | C(k) L1: 0.017 ± 0.002 -> 0.453 ± 0.002 | Hurt |
| Add state flow regularization with λ=10^-5 (ablation) | DB, Δ log F · Permutation n=20 | C(k) L1: 0.453 ± 0.002 -> 0.017 ± 0.002 | Helped |
| Use a smaller regularization coefficient, λ=10^-5 rather than λ=10^-3 (ablation) | DB, Δ log F · Permutation n=20 | C(k) L1: 0.041 ± 0.002 -> 0.017 ± 0.002 | Helped |
| Use a smaller regularization coefficient, λ=10^-5 rather than λ=10^-3 (ablation) | DB, Δ log F · Permutation n=20 | E[nτ]: 3.23 ± 0.00 -> 7.55 ± 0.50 | Hurt |
| Use SDB instead of DB loss (ablation) | Δ log F with λ=10^-5 · Permutation n=20 | C(k) L1: 0.017 ± 0.002 -> 0.014 ± 0.001 | Helped |
| Use SDB instead of DB loss (ablation) | Δ log F with λ=10^-5 · Permutation n=20 | E[nτ]: 7.55 ± 0.50 -> 7.31 ± 0.07 | Helped |
| Use ΔF-scale rather than Δ log F-scale loss (ablation) | Trainable-backward-policy GFlowNets · 20×20×20×20 hypergrid | sampling accuracy | Hurt |
| Add state flow regularization (ablation) | DB and SDB with Δ log F scale · 20×20×20×20 hypergrid | mean trajectory length | Helped |
| Use ΔF-scale loss instead of Δ log F-scale loss (ablation) | Trainable-backward-policy GFlowNets · Permutation n=20 | E[nτ]: 7.55 ± 0.50 -> 2.00 ± 0.00 | Helped |
| Use ΔF-scale rather than Δ log F-scale loss (ablation) | Trainable-backward-policy GFlowNets · Permutation n=8 | C(k) L1: 0.005 ± 0.001 -> 0.215 ± 0.198 | Hurt |
Surprising findings
- Standard DB in flow scale appears stable in experiments, with small expected trajectory length even when backward policy is trained.
- Losses in log-flow scale without regularization can still give small trajectory length on the small 7x7 hypergrid, but fail on larger grids.
Limitations
- Fixed backward policies can yield very large expected trajectory length and are hard to tune manually.
- Regularization strength lambda trades off trajectory length against reward matching accuracy.
- Experiments limited to small synthetic discrete environments (hypergrids, permutations).
Experimental setup
Datasets: hypergrid environment (7x7 and 20x20x20x20) permutation environment S_n (n=8, 20)
Benchmarks: hypergrid total variation permutation fixed-point distribution L1 error
Metrics: total variation distance L1 distance C(k) relative error of mean reward log Z error expected trajectory length
Baselines: DB SDB
Builds on: GFlowNets detailed balance trajectory balance stable losses for non-acyclic GFlowNets
Closest papers by meaning
- A Theory of Non-acyclic Generative Flow Networks (AAAI 2024 · similarity 0.94)
- Stable GFlowNets with Probabilistic Guarantees (NeurIPS 2026 · similarity 0.91)
- Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks (ICLR 2025 · similarity 0.91)
- Pessimistic Backward Policy for GFlowNets (NeurIPS 2024 · similarity 0.91)
- Flow Factorization for Efficient Generative Flow Networks (AAAI 2025 · similarity 0.90)
- Pre-Training and Fine-Tuning Generative Flow Networks (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.