Research map / 3D vision / 3D generation
Nabla-R2D3: Effective and Efficient 3D Diffusion Alignment with 2D Rewards
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Nabla-R2D3 finetunes 3D-native diffusion models (DiffSplat, GaussianCube) with 2D reward models (aesthetic, HPSv2, normal-estimator, depth-normal consistency) via a Nabla-GFlowNet-style gradient-matching loss averaged over rendered views.
Problem
Aligning 3D-native diffusion models with human preferences (aesthetics, text alignment, geometry) is hard because high-quality 3D reward models and 3D preference data are scarce, and lifting-from-2D or vanilla RL/reward finetuning methods are unstable, overfit, or reward-hack.
Approach
Extend Nabla-GFlowNet's score-matching-like gradient-informed consistency losses to 3D by defining the 3D reward as the expectation of 2D rewards over rendered camera views, so 3D-native diffusion models can be finetuned using only 2D reward models; also proposes geometry rewards (normal-estimator consistency and depth-normal consistency).
Key results
- On Aesthetic Score, Nabla-R2D3 reaches reward 6.44 vs 5.82 (ReFL), 5.51 (DRaFT), 4.54 (DDPO) and base 4.72, but with FID 217.89 (DDPO 172.95, base 55.26).
- On HPSv2, Nabla-R2D3 gets reward 27.85 (x1e-2) vs base 22.86, with CLIP-Sim 35.35 vs base 34.58 and FID 131.38 (lower than ReFL 274.44 and DRaFT 224.64).
- Nabla-R2D3 takes about 8 seconds inference per object after finetuning versus about 5 minutes for 3D-SDS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Nabla-R2D3 instead of ReFL | DiffSplat-PixArt-Σ · Aesthetic Score | Reward: 5.82 -> 6.44 | Helped |
| Use Nabla-R2D3 instead of DDPO | DiffSplat-PixArt-Σ · Aesthetic Score | FID: 172.95 -> 217.89 | Hurt |
| Use Nabla-R2D3 instead of DDPO | DiffSplat-PixArt-Σ · Aesthetic Score | CLIP-Sim: 32.64 -> 25.86 | Hurt |
| Use Nabla-R2D3 instead of DRaFT | DiffSplat-PixArt-Σ · HPSv2 | Reward: 32.65 -> 27.85 | Hurt |
| Use Nabla-R2D3 instead of DDPO | DiffSplat-PixArt-Σ · HPSv2 | FID: 69.59 -> 131.38 | Hurt |
| Use Nabla-R2D3 instead of ReFL | DiffSplat-PixArt-Σ · HPSv2 | CLIP-Sim: 32.40 -> 35.35 | Helped |
| Use Nabla-R2D3 instead of DRaFT | DiffSplat-PixArt-Σ · Normal Estimator | Reward: 91.11 -> 92.03 | Helped |
| Use Nabla-R2D3 instead of DDPO | DiffSplat-PixArt-Σ · Normal Estimator | FID: 63.93 -> 104.45 | Hurt |
| Use Nabla-R2D3 instead of DDPO | DiffSplat-PixArt-Σ · Depth-Normal Consistency | Reward: 87.99 -> 89.53 | Helped |
| Use Nabla-R2D3 instead of DDPO | DiffSplat-PixArt-Σ · Depth-Normal Consistency | FID: 69.12 -> 99.01 | Hurt |
| Increase reward temperature β (ablation) | Nabla-R2D3 · Aesthetic Score | Reward | Helped |
| Increase reward temperature β (ablation) | Nabla-R2D3 · Aesthetic Score | FID | Hurt |
Surprising findings
- Nabla-R2D3 does not achieve the best FID on Aesthetic Score or HPSv2 (DDPO attains better FID and CLIP-Sim there), so prior preservation is not uniformly superior.
- Reward gains come with reward hacking and unnatural artifacts (e.g., animal feet) if finetuned too long.
Limitations
- Gradient computation during forward pass is expensive.
- Inherits lifting-from-2D limitations such as no supervision of inner structures.
- Only parameter-level alignment; prompt-based alignment not explored.
Experimental setup
Datasets: G-Objaverse prompts LAION-Aesthetic HPDv2
Benchmarks: G-Objaverse prompts (60 unseen prompts)
Metrics: reward multi-view FID multi-view CLIP similarity
Baselines: DDPO ReFL DRaFT 3D-SDS DreamReward
Builds on: Nabla-GFlowNet DDPO DRaFT DreamFusion
Closest papers by meaning
- Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets (ICLR 2025 · similarity 0.89)
- Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards (CVPR 2025 · similarity 0.88)
- PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference (NeurIPS 2024 · similarity 0.87)
- Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL (NeurIPS 2026 · similarity 0.86)
- Acc3D: Accelerating Single Image to 3D Diffusion Models via Edge Consistency Guided Score Distillation (CVPR 2025 · similarity 0.86)
- DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.