Research map / Image generation / Editing and style transfer
ProtoSnap: Prototype Alignment For Cuneiform Signs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Unsupervised ProtoSnap aligns skeleton-based font prototypes to photographed cuneiform signs using fine-tuned diffusion features, and uses the alignments to condition ControlNet for synthetic data that boosts cuneiform OCR, especially on rare signs.
Problem
Recovering the fine-grained internal stroke configuration of photographed cuneiform signs, which prior automated methods treat as categorical, and using it to improve sign recognition especially for rare signs.
Approach
Snap a four-keypoint-per-stroke skeleton prototype to target sign images via best-buddies matches on a fine-tuned Stable Diffusion DIFT 4D similarity volume, followed by per-stroke projective refinement with saliency, similarity and regularization losses; use the aligned skeletons as ControlNet conditions to generate structurally correct synthetic training data.
Key results
- ProtoSnap full achieves F1@20 of 27.14% versus 16.14% for the best generic baseline (DIFT) on the expert-annotated alignment test set.
- Adding CN-generated data raises CSDD accuracy from 58.43% to 64.14% and rare-sign accuracy from 25.84% to 53.17%.
- In a user survey of 12 assyriologists, the aligned overlay was preferred 65% of the time over the unaligned overlay.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ProtoSnap global alignment and local refinement | Cuneiform prototype-to-image alignment · Cuneiform alignment test set | F1@20: 16.12% -> 27.14% | Helped |
| Add ProtoSnap global alignment and local refinement | Cuneiform prototype-to-image alignment · Cuneiform alignment test set | F1@30: 28.19% -> 42.09% | Helped |
| Add ProtoSnap global alignment and local refinement | Cuneiform prototype-to-image alignment · Cuneiform alignment test set | F1@40: 37.93% -> 52.43% | Helped |
| Add local refinement after global alignment | ProtoSnap · Cuneiform alignment test set | F1@20: 21.31% -> 27.14% | Helped |
| Add synthetic data generated with text-conditioned SD | ResNet18 trained on CSDD · CSDD, rare signs | Accuracy: 25.84% -> 38.56% | Helped |
| Use structurally conditioned CN-generated data instead of text-conditioned SD-generated data | ResNet18 trained on CSDD · CSDD, rare signs | Accuracy: 38.56% -> 53.17% | Helped |
| Use base Stable Diffusion instead of fine-tuned SD (ablation) | ProtoSnap · Cuneiform alignment test set | F1@40: 52.43% -> 41.25% | Hurt |
| Use all DIFT best-matching correspondences instead of best-buddy correspondences (ablation) | ProtoSnap · Cuneiform alignment test set | F1@40: 52.43% -> 52.76% | No clear effect |
| Add the similarity loss (ablation) | ProtoSnap · Cuneiform alignment test set | F1@40: 53.19% -> 52.43% | Hurt |
| Add the saliency loss (ablation) | ProtoSnap · Cuneiform alignment test set | F1@40: 50.08% -> 52.43% | Helped |
| Add the regularization loss (ablation) | ProtoSnap · Cuneiform alignment test set | F1@40: 47.41% -> 52.43% | Helped |
Surprising findings
- Removing the similarity loss slightly improves alignment metrics (F1@20 27.37% vs 27.14%), which the authors attribute to a qualitative trade-off.
- Plain DIFT + RANSAC performs worse than DIFT alone at all thresholds.
Limitations
- Fails when the scan shows a structurally different sign variant than the prototype.
- Fails under extreme deformations or on low-quality or corrupt tablet scans.
- Requires manually annotated skeleton prototypes per sign.
Experimental setup
Datasets: eBL classification dataset CSDD
Benchmarks: ProtoSnap expert-annotated alignment test set (272 images, 25 signs) CSDD sign classification
Metrics: F1@20 F1@30 F1@40 precision recall accuracy balanced accuracy rare-sign accuracy
Baselines: SIFT + RANSAC DINOv2 DINOv2 + RANSAC DIFT DIFT + RANSAC PoseAnything CSDD ResNet18 classifier SD-generated synthetic data
Builds on: DIFT Stable Diffusion ControlNet best-buddies correspondences
Closest papers by meaning
- OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones? (ICLR 2025 · similarity 0.86)
- Component-Level Segmentation for Oracle Bone Inscription Decipherment (AAAI 2025 · similarity 0.82)
- AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ (ICLR 2024 · similarity 0.82)
- Towards Automated Chinese Ancient Character Restoration: A Diffusion-Based Method with a New Dataset (AAAI 2024 · similarity 0.81)
- VecGlypher: Unified Vector Glyph Generation with Language Models (CVPR 2026 · similarity 0.81)
- VSNet: Focusing on the Linguistic Characteristics of Sign Language (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.