Research map / Probabilistic and scientific modeling / Variational and generative sampling

Spherical Flows for Sampling Categorical Data

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Flow matching and flow sampling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a von Mises-Fisher conditional path on the sphere for continuous generative modeling of discrete sequences, reducing the velocity to a scalar ODE and enabling ODE and predictor-corrector sampling from a single posterior-trained model.

Problem

Learning generative models for discrete sequences in continuous embedding space; prior continuous methods operate in Euclidean space or on the simplex, and the conditional velocity for a sphere-based noise process is intractable in general.

Approach

Use the radially symmetric vMF distribution as the conditional path on S^{d-1}; radial symmetry reduces the continuity equation to a 1D flux ODE in cosine similarity, giving a closed-form (precomputable) conditional velocity and score. The only learned object is the per-position token posterior, trained with cross-entropy.

Key results

• On Sudoku-Extreme, vMF with predictor-corrector sampling reaches 74.5% validity versus 38.8% for MDM (p=1) and 69.1% for VP.
• On LM1B under PC with self-conditioning, vMF achieves gen. PPL 33.8 at entropy 4.21, versus 83.1 for VP at the same floor.
• On GSM8K (TinyGSM training), vMF with self-conditioning and PC reaches 43.5% accuracy, above MDLM and S-FLM (18.0%) but below AR greedy (63.3%).

Surprising findings

• Under plain ODE sampling on LM1B, vMF does not lead; VP attains the lowest gen. PPL, and the advantage of vMF appears only with predictor-corrector sampling.
• Self-conditioned ODE sampling on OpenWebText sheds entropy below the data entropy as NFE grows, a effect also seen for VP and Geodesic.

Limitations

• Conditional velocity and radial CDF must be computed numerically, requiring care at high concentration and embedding dimension to avoid underflow.
• Still below autoregressive baselines on GSM8K and LM1B perplexity under ODE sampling.
• Gap between vMF and other paths is not fully explained by the authors.

Experimental setup

Datasets: Sudoku-Extreme LM1B OpenWebText TinyGSM

Benchmarks: Sudoku-Extreme LM1B OpenWebText GSM8K

Metrics: validity generation perplexity entropy accuracy

Baselines: MDLM AR Plaid LangFlow SEDD Duo DFM Diagonal FLM VP VE geodesic (slerp)

Builds on: CDCD Riemannian Flow Matching MDLM LangFlow DiT

Closest papers by meaning

• Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation (NeurIPS 2026 · similarity 0.87)
• Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization (ICLR 2024 · similarity 0.87)
• Fisher Flow Matching for Generative Modeling over Discrete Data (NeurIPS 2024 · similarity 0.86)
• Discrete Neural Flow Samplers with Locally Equivariant Transformer (NeurIPS 2025 · similarity 0.86)
• Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025 · similarity 0.86)
• Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.86)