Research map / Image generation / Autoregressive and latent generation
Latent Representation Matters: Human-like Sketches in One-shot Drawing Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies how six representational regularizers (KL, VQ, classification, prototype-based, SimCLR, Barlow twins) shape the latent space of one-shot latent diffusion models, comparing their sketches and saliency maps against humans on Omniglot and QuickDraw-FS.
Problem
One-shot drawing: generating recognizable yet original new-category variations from a single exemplar, where generative models still lag humans; the role of representational inductive biases in the latent space is unexplored.
Approach
Regularize the autoencoder latent space of LDMs with inductive biases borrowed from one-shot classification (prototype-based, Barlow twins, SimCLR, classification) and evaluate them against humans using the originality vs. recognizability framework and a new gradient-based feature importance map method compared with psychophysics saliency maps.
Key results
- Prototype-based regularization reaches minimal distance to human of 0.04 (beta=5e2) versus 0.15 for classification regularization.
- Barlow regularization reaches minimal distance to human of 0.08 versus 0.12 for SimCLR.
- Prototype-based regularizer gives feature importance maps with highest Spearman correlation to humans, significantly above all other regularizers (p<1e-3); human consistency is 0.88.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use prototype-based rather than classification regularization | LDM · QuickDraw-FS | distance to human: 0.15 -> 0.04 | Helped |
| Use Barlow rather than SimCLR regularization | LDM · QuickDraw-FS | distance to human: 0.12 -> 0.08 | Helped |
| Add prototype-based regularization to Barlow regularization (ablation) | LDM · QuickDraw-FS | recognizability | Helped |
| Add prototype-based regularization to KL regularization (ablation) | LDM · QuickDraw-FS | recognizability | Helped |
| Add prototype-based regularization to VQ regularization (ablation) | LDM · QuickDraw-FS | human-likeness | No clear effect |
| Add prototype-based regularization to SimCLR regularization (ablation) | LDM · QuickDraw-FS | human-likeness | No clear effect |
| Increase regularization weight β from low values toward an intermediate value (ablation) | LDM with each tested regularizer · QuickDraw-FS | recognizability | Mixed |
| Increase regularization weight β from low values toward an intermediate value (ablation) | LDM with each tested regularizer except VQ · QuickDraw-FS | originality | Mixed |
| Use an optimally weighted regularizer rather than no regularization (ablation) | LDM · QuickDraw-FS | alignment with human performance | Helped |
| Use prototype-based regularization rather than other tested regularizers | LDM · QuickDraw-FS | Spearman correlation with human feature importance maps | Helped |
| Use Barlow regularization rather than KL, classification, or SimCLR regularization | LDM · QuickDraw-FS | Spearman correlation with human feature importance maps | Helped |
Surprising findings
- Standard KL and VQ regularizers, used in Stable Diffusion, are not the best performers for one-shot drawing.
- VQ-regularized LDM produces irrelevant feature importance maps, possibly due to non-differentiability of quantization.
Limitations
- Only six representational inductive biases tested out of many available.
- Originality and recognizability critic networks may not align with human perceptual judgments.
- Images downsampled to 48x48 and models are small custom architectures.
Experimental setup
Datasets: Omniglot QuickDraw-FS Quick, Draw!
Benchmarks: QuickDraw-FS Omniglot
Metrics: recognizability (one-shot classifier accuracy) originality (normalized distance in self-supervised feature Spearman rank correlation of feature importance maps with hu distance to human data point
Baselines: Non-regularized LDM Pixel-space diffusion model KL-regularized LDM VQ-regularized LDM Classification-regularized LDM SimCLR-regularized LDM
Builds on: Latent Diffusion Models Prototypical Networks Barlow Twins SimCLR classifier-free guidance originality vs. recognizability framework
Closest papers by meaning
- An analytic theory of creativity in convolutional diffusion models (ICML 2025 · similarity 0.89)
- It's All About Your Sketch: Democratising Sketch Control in Diffusion Models (CVPR 2024 · similarity 0.87)
- The Hidden Language of Diffusion Models (ICLR 2024 · similarity 0.86)
- NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models (CVPR 2024 · similarity 0.86)
- PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference (NeurIPS 2024 · similarity 0.85)
- Imagination Helps Visual Reasoning, But Not Yet in Latent Space (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.