AI Paper Atlas

Research map / Detection and segmentation / Open-set and few-shot detection

CoMix: A Comprehensive Benchmark for Multi-Task Comic Understanding

Emanuele Vivoli, Marco Bertini, Dimosthenis Karatzas

NeurIPS 2024 · Detection and segmentation › Open-set and few-shot detection · approach: Object-level representation learning · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CoMix, a multi-task comic understanding benchmark with 100 books (manga, American, French) annotated for detection, speaker identification, character re-ID, reading order, character naming and dialog generation, plus a new Hybrid Dialog Score metric.

Problem

Existing comic datasets and metrics cover isolated tasks, are small or single-style (mostly manga), and lack benchmarks for character naming and dialog generation (diarization).

Approach

A multi-style benchmark built by extending three existing datasets (PopManga, DCM, eBDtheque) with dense multi-task annotations plus a newly curated set of American golden-age comics selected by a shared-to-unique character ratio, and a Hybrid Dialog Score combining edit distance and character-name ANLS.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use the closest-character heuristic instead of Magi for speaker identificationSpeaker identification · CoMix averageR@#text: 27,9 vs 38,4Improved
Use the closest-character heuristic instead of Magi for speaker identificationSpeaker identification · PopR@#text: 37,3 vs 57,2Hurt
Use DINOv2 features instead of Magi for character re-identificationCharacter re-identification · CoMix validationAMIImproved
Use Magi trained on manga for detection outside its manga domainComic-style detection · CoMixcharacter recognition performanceHurt

Experimental setup

Datasets: PopManga Digital Comic Museum (DCM) eBDtheque Manga109

Benchmarks: CoMix

Metrics: mAP@0.5 R@100 R@#text AMI NMI ANLS Hybrid Dialog Score edit distance

Baselines: GroundingDINO Faster R-CNN YOLOv8 DASS Magi DINOv2 CLIP GPT-4

Builds on: GroundingDINO Magi DASS RelationFormer

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy