Research map / Video / Video segmentation and tracking
Learning Inclusion Matching for Animation Paint Bucket Colorization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an inclusion-matching pipeline with a coarse optical-flow color warping stage for animation paint bucket colorization, plus a synthetic PaintBucket-Character dataset.
Problem
Automating paint bucket colorization of line art in hand-drawn animation, where occlusion, wrinkles and large deformations break strict segment-to-segment correspondences used by existing segment matching methods.
Approach
Instead of exact segment-to-segment matching, predict for each target segment which large reference color region (index label) it is included in; combine RAFT-based coarse color warping, deformable-conv feature alignment, CLIP line features, and a multiplex transformer over super-pixel tokens.
Key results
- On the 3D rendered test set, the proposed method reaches 0.8266 segment accuracy versus 0.6634 for AnT (Cadmium) and 0.7133 for fine-tuned RAFT.
- On the hand-drawn test set, segment accuracy is 0.8593 versus 0.7224 for AnT (Cadmium).
- Applying the inclusion matching pipeline to AnT raises its 3D test accuracy from 0.7450 to 0.8063.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PaintBucket-Character rather than AnimeRun for training (ablation) | Ours · PaintBucket-Character 3D rendered test set | Acc: 0.7359 -> 0.8266 | Helped |
| add CLIP image encoder (ablation) | Ours · PaintBucket-Character 3D rendered test set | Acc: 0.8180 -> 0.8266 | Helped |
| add CLIP image encoder (ablation) | Ours · PaintBucket-Character hand-drawn test set | Acc: 0.8489 -> 0.8593 | Helped |
| add RAFT coarse color warping (ablation) | Ours · PaintBucket-Character 3D rendered test set | Acc: 0.8001 -> 0.8266 | Helped |
| add RAFT coarse color warping (ablation) | Ours · PaintBucket-Character hand-drawn test set | Acc: 0.8446 -> 0.8593 | Helped |
| use deformable convolution instead of original convolution (ablation) | Ours · PaintBucket-Character 3D rendered test set | Acc: 0.8247 -> 0.8266 | Helped |
| use deformable convolution instead of original convolution (ablation) | Ours · PaintBucket-Character hand-drawn test set | Acc: 0.8525 -> 0.8593 | Helped |
| add inclusion matching (ablation) | Ours · PaintBucket-Character 3D rendered test set | Acc: 0.7637 -> 0.8266 | Helped |
| add inclusion matching (ablation) | Ours · PaintBucket-Character hand-drawn test set | Acc: 0.8348 -> 0.8593 | Helped |
| add inclusion matching (ablation) | AnT · PaintBucket-Character 3D rendered test set | Acc: 0.7450 -> 0.8063 | Helped |
| add inclusion matching and RAFT coarse warping (ablation) | AnT · PaintBucket-Character 3D rendered test set | Acc: 0.7450 -> 0.8177 | Helped |
Surprising findings
- AnimeRun's segment matching labels are ambiguous due to one-to-many correspondences and anti-aliasing, making it unsuitable as a paint bucket benchmark.
Limitations
- Real hand-drawn test set is small (200 frames, 20 clips) and the training data is mostly synthetic 3D renders.
- Dependence on optical flow estimated from line art for the coarse warping stage.
- Segments with no match above the threshold 0.2 are left unfilled.
Experimental setup
Datasets: PaintBucket-Character AnimeRun MPI-Sintel
Benchmarks: PaintBucket-Character 3D rendered test set PaintBucket-Character hand-drawn test set
Metrics: Acc Acc-Thres Pix-Acc Pix-F-Acc Pix-B-MIoU
Baselines: ControlNet AnT (AnimeRun) AnT (Cadmium) RAFT RAFT (F) RAFT (C)
Builds on: AnT RAFT CLIP Multiplex transformer (LoFTR) ControlNet
Closest papers by meaning
- MangaNinja: Line Art Colorization with Precise Reference Following (CVPR 2025 · similarity 0.87)
- Image Referenced Sketch Colorization Based on Animation Creation Workflow (CVPR 2025 · similarity 0.87)
- AniDoc: Animation Creation Made Easier (CVPR 2025 · similarity 0.86)
- ChromaFusionNet (CFNet): Natural Fusion of Fine-Grained Color Editing (AAAI 2024 · similarity 0.83)
- SSIMBaD: Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization (NeurIPS 2025 · similarity 0.82)
- Automatic Controllable Colorization via Imagination (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.