Research map / Reinforcement learning / Visual and representation RL
DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning
Calvin Luo, Chen Sun, Shuran Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DF-ExpEnse exploration (candidate set plus critic-ensemble UCB selection) to vanilla DSRL | DSRL · RoboMimic (Can, Square, Tool Hang) and OpenAI Gym | success rate / reward vs timesteps | Improved |
| add DF-ExpEnse to DSRL on RoboMimic Lift | DSRL · RoboMimic Lift | success rate | No clear effect |
| add DF-ExpEnse to ResFiT | ResFiT · DexMimicGen (Can Sort, Box Cleanup, Coffee) | success rate vs timesteps | Improved |
| select the action with max Q (Max-Q, alpha=0) from sampled candidates instead of vanilla sampling | DSRL · RoboMimic manipulation tasks | success rate | Hurt |
| add critic disagreement term (alpha=0.5) to action selection instead of Max-Q (ablation) | DF-ExpEnse · RoboMimic and Gym tasks | sample efficiency | Improved |
| add BC-SR (candidates from initial pretrained policy, p=1) (ablation) | DF-ExpEnse · RoboMimic Can and Square | sample efficiency | Improved |
| add BC-SR to Max-Q (ablation) | Max-Q · RoboMimic Can and Square | sample efficiency | Improved |
| add fleet normalization versus isolated agents in the same fleet (ablation) | DF-ExpEnse · RoboMimic and OpenAI Gym | sample efficiency | Improved |
| increase critic ensemble size from 2 to 10 (ablation) | DF-ExpEnse · RoboMimic Can | sample efficiency | Improved |
| increase critic ensemble size from 5 to 10 (ablation) | DF-ExpEnse · RoboMimic Can | sample efficiency | No clear effect |
| increase action candidate sample size from 3 to 5 or 7 (ablation) | DF-ExpEnse · RoboMimic Can | sample efficiency | No clear effect |
| increase fleet size toward 4 from smaller fleets (ablation) | DF-ExpEnse · RoboMimic Can | sample efficiency | Improved |
Closest papers by meaning
- ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Tasks (ICLR 2024 · similarity 0.87)
- Behavioral Exploration: Learning to Explore via In-Context Adaptation (ICML 2025 · similarity 0.87)
- Score Regularized Policy Optimization through Diffusion Behavior (ICLR 2024 · similarity 0.87)
- FASTER: Value-Guided Sampling for Fast RL (NeurIPS 2026 · similarity 0.87)
- Self-Improving Embodied Foundation Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
