Research map / Reinforcement learning / Visual and representation RL
Deep Generative Models with Learnable Knowledge Constraints
Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, LIANHUI Qin, Xiaodan Liang, Haoye Dong, Eric P Xing
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a jointly learned human-part consistency constraint | Pose-conditioned image generation · DeepFashion pose-conditioned image generation | SSIM: 0.676 -> 0.727 | Improved |
| Add a jointly learned human-part consistency constraint | Pose-conditioned image generation · DeepFashion pose-conditioned image generation | Human preference: 0.03 -> 0.77 | Improved |
| Use a fixed pretrained constraint instead of jointly learning it (ablation) | Pose-conditioned image generation · DeepFashion pose-conditioned image generation | SSIM: 0.676 -> 0.679 | No clear effect |
| Use a fixed pretrained constraint instead of jointly learning it (ablation) | Pose-conditioned image generation · DeepFashion pose-conditioned image generation | Human preference: 0.03 -> 0.12 | Improved |
| Replace q samples with pθ samples when learning the constraint (ablation) | Pose-conditioned image generation · DeepFashion pose-conditioned image generation | SSIM: 0.727 -> 0.716 | Hurt |
| Add a learned infilling-content matching constraint | Template-guided sentence generation · IMDB template-guided sentence generation | Perplexity: 30.30 -> 28.69 | Improved |
| Add a learned infilling-content matching constraint | Template-guided sentence generation · IMDB template-guided sentence generation | Human preference: 0.19 -> 0.24 | Improved |
| Add a binary discriminator instead of a structured matching constraint (ablation) | Template-guided sentence generation · IMDB template-guided sentence generation | Perplexity: 30.30 -> 30.01 | No clear effect |
| Add a binary discriminator instead of a structured matching constraint (ablation) | Template-guided sentence generation · IMDB template-guided sentence generation | Human preference: 0.19 -> 0.20 | No clear effect |
| Learn the constraint directly by fitting it to current model samples (ablation) | Template-guided sentence generation · IMDB template-guided sentence generation | Perplexity: 30.30 -> 31.27 | Hurt |
| Learn the constraint directly by fitting it to current model samples (ablation) | Template-guided sentence generation · IMDB template-guided sentence generation | Human preference: 0.19 -> 0.15 | Hurt |
Closest papers by meaning
- Adaptive Divergence Regularized Policy Optimization for Fine-tuning Generative Models (NeurIPS 2025 · similarity 0.86)
- Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization (NeurIPS 2026 · similarity 0.85)
- David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training (ICML 2025 · similarity 0.85)
- Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences (NeurIPS 2024 · similarity 0.85)
- How Realistic Is Your Synthetic Data? Constraining Deep Generative Models for Tabular Data (ICLR 2024 · similarity 0.85)
- GRAM: A Generative Foundation Reward Model for Reward Generalization (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
