Research map / Vision-language and multimodal / Cross-modal learning
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train with cross-modal vectorization/rasterization instead of the strongest single-view self-supervised baseline | sketch representations · QuickDraw recognition, image, frozen | Top-1 accuracy: 66.8% -> 71.9% | Helped |
| pre-train with cross-modal vectorization/rasterization instead of Sketch-BERT | sketch representations · QuickDraw recognition, vector, frozen | Top-1 accuracy: 65.6% -> 67.2% | Helped |
| pre-train with cross-modal vectorization/rasterization instead of the strongest listed baseline | sketch representations · QuickDraw recognition, image, 1% labels | Top-1 accuracy: 47.3% -> 51.2% | Helped |
| pre-train with cross-modal vectorization/rasterization instead of CPC | handwriting representations · IAM offline handwriting, 1% labels | Word Recognition Accuracy: 29.1% -> 38.5% | Helped |
| use absolute rather than offset coordinates in vectorization decoding (ablation) | vectorization pre-training · QuickDraw recognition, image | Top-1 accuracy: 69.5% -> 71.9% | Helped |
| use absolute rather than offset coordinates in rasterization encoding (ablation) | rasterization pre-training · QuickDraw recognition, vector | Top-1 accuracy: 67.1% -> 67.2% | No clear effect |
| use GRU rather than LSTM or Transformer as the vectorization decoder (ablation) | vectorization pre-training · QuickDraw recognition, image | Top-1 accuracy: 70.7% -> 71.9%; 68.6% -> 71.9% | Helped |
| use Transformer rather than LSTM or GRU as the rasterization encoder (ablation) | rasterization pre-training · QuickDraw recognition, vector | Top-1 accuracy: 66.7% -> 67.2%; 66.1% -> 67.2% | Helped |
| add two-way cross-modal translation (ablation) | vectorization/rasterization pre-training · QuickDraw recognition, image and vector | Top-1 accuracy: 71.9% -> 70.3%; 67.2% -> 66.1% | Hurt |
| add attention to the vectorization decoder (ablation) | vectorization pre-training · QuickDraw recognition, image | Top-1 accuracy: 71.9% -> 68.0% | Hurt |
| add an adversarial loss to rasterization training (ablation) | rasterization pre-training · representation learning | downstream performance | Mixed |
| replace deterministic cross-modal encoder-decoder with a VAE-based design (ablation) | cross-modal pre-training · sketch representation learning | downstream performance | Hurt |
Closest papers by meaning
- MetaWriter: Personalized Handwritten Text Recognition Using Meta-Learned Prompt Tuning (CVPR 2025 · similarity 0.86)
- SketchINR: A First Look into Sketches as Implicit Neural Representations (CVPR 2024 · similarity 0.86)
- DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models (NeurIPS 2023 · similarity 0.86)
- Sketch2Diagram: Generating Vector Diagrams from Hand-Drawn Sketches (ICLR 2025 · similarity 0.85)
- Efficient Sketches for Training Data Attribution and Studying the Loss Landscape (NeurIPS 2024 · similarity 0.85)
- TIPS: Text-Image Pretraining with Spatial awareness (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.