Research map / Image generation / Autoregressive and latent generation
Learning to Infer Graphics Programs from Hand-Drawn Images
Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, Josh Tenenbaum
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine the neural proposal network with SMC rather than using either alone (ablation) | Graphics spec parsing · Scenes with more objects than the training data | Generalization to cluttered scenes | Improved |
| Use the neural network alone with beam search instead of neural proposals plus SMC (ablation) | Graphics spec parsing · Scenes with many more objects than training data | Generalization to cluttered scenes | Hurt |
| Use SMC alone instead of neural proposals plus SMC (ablation) | Graphics spec parsing · Scenes with many more objects than training data | Generalization to cluttered scenes | Hurt |
| Replace command-by-command spec inference with a one-shot CNN-to-LSTM spec decoder (ablation) | Graphics spec parsing · Cluttered scenes with many objects | Parsing ability | Hurt |
| Add a learned prior over synthesized programs to rerank candidate specs | Neurally guided SMC · 100 real hand-drawn figures | Top-1 accuracy: 63% to 67% | Improved |
| Use a learned search policy instead of sending the whole synthesis problem to Sketch | Graphics program synthesis · Graphics program synthesis problems | Median search time: 274 sec -> 28 sec | Improved |
| Use a learned search policy instead of the DC component-prediction baseline | Graphics program synthesis · Graphics program synthesis problems | Median search time: 187 sec -> 28 sec | Improved |
| Regress directly from images to programs instead of factoring through specs and synthesis (ablation) | Graphics program induction · Graphics program synthesis problems | Timeouts (1 hr): 2% -> 94% | Hurt |
| Use direct image-to-program regression rather than the factored system (ablation) | Graphics program induction · Graphics program synthesis problems | Median search time: 28 sec -> 63 sec | Hurt |
| Rerank candidate specs by program simplicity and structure (ablation) | Neural spec predictions · Hand-drawn figures | Top-1 accuracy: 63% to 67% | Mixed |
Closest papers by meaning
- SketchEmbedNet: Learning Novel Concepts by Imitating Drawings (ICML 2021 · similarity 0.88)
- Deep Learning for Free-Hand Sketch: A Survey (TPAMI 2022 · similarity 0.87)
- AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches (AAAI 2019 · similarity 0.86)
- Scribbler: Controlling Deep Image Synthesis With Sketch and Color (CVPR 2017 · similarity 0.86)
- SketchINR: A First Look into Sketches as Implicit Neural Representations (CVPR 2024 · similarity 0.85)
- Learning to Infer Generative Template Programs for Visual Concepts (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
