Research map / Image generation / Autoregressive and latent generation
Integrating Symbolic Reasoning into Neural Generative Models for Design Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPRING, which embeds a SampleSearch-based symbolic spatial reasoning module, driven by an RNN implicit-preference distribution, into a Stable Diffusion inpainting pipeline to generate interior designs that guarantee user spatial constraints.
Problem
Automated design generation must satisfy explicit user spatial specifications and implicit aesthetic/utility preferences, but neural text-to-image models fail to follow precise constraints and symbolic constraint solvers cannot perceive images or capture aesthetics.
Approach
Sample object bounding boxes via iterative range-halving decisions, using an RNN for implicit preferences and zeroing out constraint-violating decisions through SampleSearch to get a backtrack-free distribution; a Stable Diffusion inpainting model then fills each box.
Key results
- SPRING guarantees constraint satisfaction by construction, so zero violations are produced, whereas text-based and scene-graph baselines fail on complex specifications.
- On synthetic preference scenarios, the spatial reasoning module outperforms GAN + CVX on preference accuracy.
- Images produced by SPRING reach image quality on par with recent image generation tools by FID and IS, per the paper's claim.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SD V1.2 pre-finetuned on inpainting tasks instead of SD V1.4 fine-tuned on COCO | Stable Diffusion VEG · VEG comparison, all backgrounds | images with one or more inpaint failures: 13 -> 8 | Helped |
| Use SD V1.2 pre-finetuned on inpainting tasks instead of SD V1.4 fine-tuned on COCO | Stable Diffusion VEG · VEG comparison, all backgrounds | images with one or more clear continuity failures: 22 -> 0 | Helped |
| Pre-finetune SD V1.2 on inpainting tasks instead of using SD V1.2 | Stable Diffusion VEG · VEG comparison, all backgrounds | images with one or more inpaint failures: 6 -> 8 | Hurt |
| Pre-finetune SD V1.2 on inpainting tasks instead of using SD V1.2 | Stable Diffusion VEG · VEG comparison, all backgrounds | images with one or more clear continuity failures: 20 -> 0 | Helped |
Limitations
- Constraints must be given in a structured propositional language rather than natural language.
- Diffusion-based filling can still produce inpainting failures and wrong objects.
- The paper evaluates mainly on interior design, with an outdoor animal instance only as a supplement.
Experimental setup
Datasets: COCO LAION-aesthetics v2 5+ ImageNet synthetic positioning datasets
Benchmarks: synthetic spatial reasoning scenarios (basic, tight, complex) FID IS human study
Metrics: constraint satisfaction accuracy preference accuracy FID Inception Score human ratings (specification satisfaction, aesthetics, backg
Baselines: Stable Diffusion SG2IM GAN + CVX GLIDE
Builds on: SampleSearch Stable Diffusion DETR LoRA sg2im
Closest papers by meaning
- MaskPLAN: Masked Generative Layout Planning from Partial Input (CVPR 2024 · similarity 0.85)
- Can Large Language Models Understand Symbolic Graphics Programs? (ICLR 2025 · similarity 0.85)
- GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models (CVPR 2026 · similarity 0.85)
- Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning (NeurIPS 2025 · similarity 0.84)
- GoT-R1: Unleashing Reasoning Capability of Autoregressive Visual Generation with Reinforcement Learning (ICLR 2026 · similarity 0.84)
- Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
