Research map / Detection and segmentation / Open-set and few-shot detection
CoMix: A Comprehensive Benchmark for Multi-Task Comic Understanding
Emanuele Vivoli, Marco Bertini, Dimosthenis Karatzas
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CoMix, a multi-task comic understanding benchmark with 100 books (manga, American, French) annotated for detection, speaker identification, character re-ID, reading order, character naming and dialog generation, plus a new Hybrid Dialog Score metric.
Problem
Existing comic datasets and metrics cover isolated tasks, are small or single-style (mostly manga), and lack benchmarks for character naming and dialog generation (diarization).
Approach
A multi-style benchmark built by extending three existing datasets (PopManga, DCM, eBDtheque) with dense multi-task annotations plus a newly curated set of American golden-age comics selected by a shared-to-unique character ratio, and a Hybrid Dialog Score combining edit distance and character-name ANLS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the closest-character heuristic instead of Magi for speaker identification | Speaker identification · CoMix average | R@#text: 27,9 vs 38,4 | Improved |
| Use the closest-character heuristic instead of Magi for speaker identification | Speaker identification · Pop | R@#text: 37,3 vs 57,2 | Hurt |
| Use DINOv2 features instead of Magi for character re-identification | Character re-identification · CoMix validation | AMI | Improved |
| Use Magi trained on manga for detection outside its manga domain | Comic-style detection · CoMix | character recognition performance | Hurt |
Experimental setup
Datasets: PopManga Digital Comic Museum (DCM) eBDtheque Manga109
Benchmarks: CoMix
Metrics: mAP@0.5 R@100 R@#text AMI NMI ANLS Hybrid Dialog Score edit distance
Baselines: GroundingDINO Faster R-CNN YOLOv8 DASS Magi DINOv2 CLIP GPT-4
Builds on: GroundingDINO Magi DASS RelationFormer
Closest papers by meaning
- Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions (NeurIPS 2024 · similarity 0.85)
- The Manga Whisperer: Automatically Generating Transcriptions for Comics (CVPR 2024 · similarity 0.85)
- ASCIIEval: Benchmarking Models' Visual Perception in Text Strings via ASCII Art (ICLR 2026 · similarity 0.84)
- ViStoryBench: Comprehensive Benchmark Suite for Story Visualization (CVPR 2026 · similarity 0.83)
- DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation (CVPR 2025 · similarity 0.83)
- MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
