Research map / Reinforcement learning / RL from feedback
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Pengcheng Jiang, Judith Yue Li, Moonkyung Ryu, Lily Hu, Kun Su, Zhong Yi Wan, Liam Hebert, Hao Peng, Jiawei Han, Dima Kuzmin, Craig Boutilier
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL-train the fan-out LM with set-level rewards | Gemma3-4B fan-out · Polyvore OAR | average score: 38.5 -> 49.1 | Improved |
| RL-train the fan-out LM with set-level rewards | Qwen3-4B fan-out · Polyvore OAR | average score: 28.1 -> 42.6 | Improved |
| RL-train the fan-out LM instead of Best-of-N sampling | Gemma3-4B · Music OAR | average score: 49.2 -> 58.1 | Improved |
| use zero-shot fan-out instead of no fan-out | Gemini-2.5-Flash · Polyvore OAR | average score: 26.1 -> 31.5 | Improved |
| Best-of-N selection with N=5 using training rewards | Gemma3-4B zero-shot · Polyvore OAR | average score: 38.5 -> 40.9 | Improved |
| distill FOLM into diffusion retriever | R4T · Polyvore OAR (Gemma3-4B) | diversity: 76.8 -> 74.3 | No clear effect |
| distill FOLM into diffusion retriever | R4T · Polyvore WSCR (Gemma3-4B) | Vendi Score: 40.5 -> 46.2 | Improved |
| distill FOLM into diffusion retriever | R4T · Polyvore WSCR (Gemma3-4B) | Recall@5K: 16.9 -> 15.0 | Hurt |
| replace autoregressive fan-out with diffusion single pass | fan-out generation · query fan-out latency | latency: 1.46 s -> 0.07 s at batch 8 | Improved |
| train with groundedness reward only (no diversity) (ablation) | FOLM RL · Polyvore OAR | training stability | Hurt |
| train with groundedness and alignment, no diversity (ablation) | FOLM RL · Polyvore OAR | training stability | Hurt |
| add diversity reward to groundedness and alignment (ablation) | FOLM RL · Polyvore OAR | training stability | Improved |
Closest papers by meaning
- RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models (ICLR 2025 · similarity 0.83)
- Learning to Rank in Generative Retrieval (AAAI 2024 · similarity 0.83)
- DITTO: Diffusion Inference-Time T-Optimization for Music Generation (ICML 2024 · similarity 0.82)
- Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (ICML 2024 · similarity 0.82)
- Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting (ICLR 2025 · similarity 0.82)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
