Research map / Detection and segmentation / Open-set and few-shot detection

Advancing Manga Analysis: Comprehensive Segmentation Annotations for the Manga109 Dataset

CVPR 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Interactive and referring segmentation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Releases MangaSeg, a segmentation-mask extension of the Manga109 dataset (10,130 pages, 6 categories) built via semi-automatic coarse annotation and iterative LoRA-SAM refinement, and benchmarks existing segmenters on it.

Problem

Manga109 only has bounding-box annotations, which are too coarse for pixel-level segmentation of B/W manga elements (panels, balloons, text, faces, bodies) because screentones and the domain gap defeat natural-image models.

Approach

Build instance-level segmentation masks for Manga109 through a semi-automatic pipeline: coarse masks from existing specialized detectors and colorization-plus-GroundedSAM for faces, then iterative human correction and LoRA fine-tuning of SAM, with IoU-based quality control against the original boxes.

Key results

• Fine-tuned SAM on MangaSeg reaches 0.954 frame/panel box AP and 0.914 balloon box AP on the test set, versus 0.354 and 0.158 for GroundedSAM.
• Text/dialog mask AP reaches 0.688 with the fine-tuned model, above GroundedSAM at 0.008.
• The fine-tuned model reports over 60% AP for all manga components relative to GroundedSAM, per the paper's claim.

Measured effects reported in the paper

Surprising findings

• Onomatopoeia segmentation remains weak (mask AP 0.272), attributed to scarcity of onomatopoeia and resulting false positives.
• Off-the-shelf GroundedSAM performs poorly on manga (e.g., 0.000 face AP), showing the domain gap to natural images.

Limitations

• Ambiguity from overlapping panels, balloons occluding frame boundaries, and missing panel boundaries.
• Difficulty annotating detailed face boundaries such as hair strands.
• Balloons annotated on inner boundaries only, limiting use for inpainting needing outer boundaries.

Experimental setup

Datasets: Manga109 DanbooRegion COMICS BCBId

Benchmarks: MangaSeg test set (927 samples)

Metrics: Box AP Mask AP Boundary AP mAP@0.5

Baselines: Chen et al. panel segmentation [33] Liu et al. balloon extraction [29] text segmentation [19] instance-aware character segmentation [25] GroundedSAM DASSDet+SAM Magi+SAM

Builds on: Segment Anything Model LoRA GroundedSAM Manga109

Closest papers by meaning

• DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation (CVPR 2025 · similarity 0.85)
• The Manga Whisperer: Automatically Generating Transcriptions for Comics (CVPR 2024 · similarity 0.84)
• Alchemist: Turning Public Text-to-Image Data into Generative Gold (NeurIPS 2025 · similarity 0.84)
• PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding (NeurIPS 2025 · similarity 0.83)
• Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model (AAAI 2025 · similarity 0.83)
• Seg2Any: Open-set Segmentation-Mask-to-Image Generation with Precise Shape and Semantic Control (NeurIPS 2025 · similarity 0.82)