Research map / Probabilistic and scientific modeling / Molecules and proteins
SketchINR: A First Look into Sketches as Implicit Neural Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SketchINR, an auto-decoder implicit neural representation that maps time and stroke stamps to xy pen coordinates, compressing vector sketches into fixed-size latents with a VAE for generation.
Problem
Existing vector sketch representations either discard temporal information (raster), scale poorly in storage (vector), or are slow and inaccurate on complex sketches (autoregressive learned vector models like SketchRNN).
Approach
Model a vector sketch as an implicit function of time and stroke stamps conditioned on a per-sketch latent code learned by an auto-decoder (DeepSDF-style), trained with a rasterised intensity-map visual loss plus an MSE loss, enabling parallel decoding and controllable abstraction.
Key results
- On FS-COCO, SketchINR reaches Chamfer distance 0.011 and R@10 15.61 versus 0.41 / 5.17 for SketchODE, where SketchRNN and CoSE fail entirely.
- On Sketchy, SketchINR achieves CD 0.008 and R@10 57.29, against 0.16 / 41.65 for SketchRNN and 0.14 / 45.82 for CoSE.
- Compression with a 64-dimensional latent per sketch gives about 60x lower bytes-per-point than PNG raster sketches and about 10x lower than vector sketches on FS-COCO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Local (per-stroke) time modelling instead of global time modelling (ablation) | SketchINR · sketch reconstruction (qualitative) | visual smoothness | Hurt |
| Variable (scheduled) smoothing factor gamma during training (ablation) | SketchINR · training time | training time reduction: reduces training time by 6.1% | No clear effect |
| Visual (intensity-map) loss plus weighted MSE loss, versus MSE alone (ablation) | SketchINR · sketch reconstruction (qualitative) | sampling robustness at different resolutions | Helped |
| Compact latent codes (64-dim) for SketchINR versus raster and vector storage | SketchINR · FS-COCO compression | bytes per point: ~60x lower BPP than PNG raster, ~10x lower than vector | Helped |
| Parallel decoding of all time and stroke points versus autoregressive decoding | SketchINR · decoding speed | decoding time: Sketchy decode 0.0007 s vs 0.11 s for SketchRNN | Helped |
Surprising findings
- SketchRNN fails completely on FS-COCO complex scene sketches, while SketchINR scales to them.
Limitations
- Optimisation-based encoding is slow to converge due to pixel-space optimisation
- Poor cross-category generalisation; generation network fails when trained on more than 2-3 categories
- Vectorisation for conditional generation performs poorly on high complexity sketches
Experimental setup
Datasets: FS-COCO Sketchy QuickDraw! Vector-MNIST
Benchmarks: FS-COCO Sketchy Quick-Draw! Vector-MNIST
Metrics: Chamfer distance recall@10 decoding time bytes per point
Baselines: SketchRNN CoSE SketchODE BezierSketch
Builds on: DeepSDF SketchRNN positional embeddings (NeRF-style) variational autoencoder
Closest papers by meaning
- Sketchformer: Transformer-Based Representation for Sketched Structure (CVPR 2020 · similarity 0.91)
- StrokeFusion: Vector Sketch Generation via Joint Stroke-UDF Encoding and Latent Sequence Diffusion (AAAI 2026 · similarity 0.90)
- It's All About Your Sketch: Democratising Sketch Control in Diffusion Models (CVPR 2024 · similarity 0.88)
- Lego Sketch: A Scalable Memory-augmented Neural Network for Sketching Data Streams (ICML 2025 · similarity 0.88)
- Sketch Down the FLOPs: Towards Efficient Networks for Human Sketch (CVPR 2025 · similarity 0.87)
- SketchFusion: Learning Universal Sketch Features through Fusing Foundation Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.