Research map / Vision-language and multimodal / Human and embodied modeling
Making Visual Sense of Oracle Bones for You and Me
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Conducts a human study on oracle bone script visual guides, proposes the TransOV metric to approximate human judgments, and introduces GenOV, a training-free pipeline using GPT-4V reasoning plus GLIGEN generation to create visual guides for oracle glyphs.
Problem
Oracle bone script glyphs are hard for modern viewers to interpret; expert-curated visual guides are laborious to create, and there is no automatic way to generate or evaluate such guides.
Approach
Treat oracle visual-guide creation as a conditional generation task (glyph + character meaning -> guide) solved without training: a VLM expands the semantics, proposes layouts, and selects among candidates generated by a layout-conditioned diffusion model, with a learned TransOV metric to score guide effectiveness.
Key results
- TransOV metric reaches 84.75% accuracy against human labels on the test set.
- GenOV guides score 89.19% TransOV versus 91.89% for expert-curated Manual guides on the test set, and beat ControlNet (CDM 0.5 at 62.16%) and text-only Stable Diffusion variants (54.05% to 78.38%).
- In the human study, the share of oracles judged incomprehensible fell from 87% (316/364) to 28% (101/364) after a visual guide was shown.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate visual guides with GenOV instead of manual curation | Oracle visual guides · Oracle visual-guide test set | TransOV accuracy: 89.19% v.s. 91.89% | No clear effect |
| Use expanded textual description oti rather than only semantic character osi for conditioning (ablation) | Stable Diffusion visual-guide generation · Oracle visual-guide test set | TransOV accuracy: 54.05% -> 72.97% | Helped |
| Condition on both osi and oti rather than only osi (ablation) | Stable Diffusion visual-guide generation · Oracle visual-guide test set | TransOV accuracy: 54.05% -> 78.38% | Helped |
| Apply ControlNet with condition scale 1.0 rather than text-only osi conditioning (ablation) | Oracle visual-guide generation · Oracle visual-guide test set | TransOV accuracy: 54.05% -> 43.24% | Hurt |
| Apply ControlNet with condition scale 0.75 rather than text-only osi conditioning (ablation) | Oracle visual-guide generation · Oracle visual-guide test set | TransOV accuracy: 54.05% -> 56.76% | Mixed |
Surprising findings
- Text-only Stable Diffusion with semantic expansion (LDM with osi and oti) outperforms prompting with the bare character alone, showing a single character is insufficient as a text condition.
- Forcing strict structural fidelity to the oracle glyph (CDM 1.0) lowers guide effectiveness, suggesting a balance between glyph fidelity and conceptual clarity is needed.
Limitations
- GPT-4V lacks object localisation ability, so Qwen-VL was needed for that step.
- Evaluation relies on a learned TransOV metric and a single human study with 28 participants.
- Oracle glyphs are highly abstract, so generated layouts may be inaccurate, requiring a verification step.
Experimental setup
Datasets: 364 oracle bone scripts crawled from two websites human study responses (9434 stage-1 and 2897 stage-2 valid trials) free-hand sketch dataset (QuickDraw/sketch data) scan-clean oracle pairs
Benchmarks: TransOV test set (37 oracles, 800 trials)
Metrics: TransOV accuracy human comprehension rate
Baselines: Manual expert-curated guides ControlNet v1.1 (CDM) Stable Diffusion text-only (LDM)
Builds on: ControlNet GLIGEN Stable Diffusion LDM layout-to-image planning DINO CLIP
Closest papers by meaning
- OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones? (ICLR 2025 · similarity 0.86)
- Component-Level Segmentation for Oracle Bone Inscription Decipherment (AAAI 2025 · similarity 0.83)
- Visual Data-Type Understanding does not emerge from scaling Vision-Language Models (ICLR 2024 · similarity 0.82)
- SALART-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images (NeurIPS 2026 · similarity 0.82)
- TABLET: A Large-Scale Dataset for Robust Visual Table Understanding (ICLR 2026 · similarity 0.81)
- VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations (ICLR 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.