Research map / Image generation / Editing and style transfer
Color Transfer with Modulated Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ModFlows, which trains per-image rectified flows into a shared uniform latent space and distills them into an EfficientNet encoder that predicts flow weights for new color transfers in RGB space.
Problem
Example-based color transfer between images, framed as an optimal transport problem, where per-pair training of flows is costly and prior neural methods rely on hard-to-obtain LUT filters or artifact-prone transfers.
Approach
Train a rectified flow per image mapping its RGB color distribution to a uniform cube, then train an encoder to predict the weights of a shallow flow for any image, so color transfer is composed as a forward content flow and an inverse style flow through the shared latent space, with no per-pair fine-tuning.
Key results
- ModFlows achieves the lowest aggregated DISTS on grayscale (0.129) and depth (0.217) content metrics, versus 0.146 for MKL on grayscale.
- Style distance for ModFlows is 0.123, second only to DAST_d (0.112), and far below NeuralPreset re-implementation (0.348).
- ModFlows encoder has average Lipschitz constant 37.26, lower than rectified flows trained from scratch (91.34).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ModFlows instead of MKL | Color transfer · Unsplash Lite 1.2.2 test set | DISTS (Grayscale): 0.146 -> 0.129 | Helped |
| Use ModFlows instead of MKL | Color transfer · Unsplash Lite 1.2.2 test set | DISTS (Depth): 0.227 -> 0.217 | Helped |
| Use ModFlows instead of PhotoWCT2 | Color transfer · Unsplash Lite 1.2.2 test set | DISTS (Edge): 0.217 -> 0.220 | Hurt |
| Use ModFlows instead of DAST_d | Color transfer · Unsplash Lite 1.2.2 test set | Style distance: 0.112 $\pm$ 0.001 -> 0.123 $\pm$ 0.001 | Hurt |
| Train rectified flows from scratch instead of using the ModFlows encoder (ablation) | ModFlows · Unsplash Lite 1.2.2 test set | DISTS (Grayscale): 0.129 -> 0.137 | Hurt |
| Train rectified flows from scratch instead of using the ModFlows encoder (ablation) | ModFlows · Unsplash Lite 1.2.2 test set | DISTS (Depth): 0.217 -> 0.250 | Hurt |
| Train rectified flows from scratch instead of using the ModFlows encoder (ablation) | ModFlows · Unsplash Lite 1.2.2 test set | Style distance: 0.123 -> 0.114 | Helped |
| Use EfficientNet B0 with dim(e)=515 and 4,767 flows instead of EfficientNet B6 with dim(e)=8195 (ablation) | ModFlows · Unsplash Lite 1.2.2 test set | DISTS (Grayscale): 0.129 -> 0.145 | Hurt |
| Use EfficientNet B0 with dim(e)=515 and 4,767 flows instead of EfficientNet B6 with dim(e)=8195 (ablation) | ModFlows · Unsplash Lite 1.2.2 test set | Style distance: 0.123 -> 0.141 | Hurt |
| Use LAB or OkLAB instead of RGB (ablation) | ModFlows · Color transfer | Results | No clear effect |
Surprising findings
- Direct rectified flows achieve lower style distance (0.114) but produce more artifacts, showing a trade-off between style and content similarity.
- The authors' re-implementation of Neural Preset with random color filters performs much worse than the original, suggesting the hand-crafted LUT filters are critical.
Limitations
- Transfer operates in RGB without semantic information, so it can cause unintended color replacements (e.g., yellow to red).
- Strength and ODE step count must be tuned to avoid artifacts for significant color changes.
- Greater dependence on the reference image than in supervised methods.
Experimental setup
Datasets: DIV2K CLIC2020 laion-art-en-colorcanny Unsplash Lite
Benchmarks: Unsplash Lite test set (1891 content-style pairs)
Metrics: DISTS on grayscale, depth, and edge content representations Wasserstein distance style distance aggregated score average Lipschitz constant
Baselines: MKL CT WCT2 PhotoWCT2 DAST PhotoNAS NeuralPreset Deep Preset Rectified flows
Builds on: rectified flow Neural Preset Monge-Kantorovich linear (MKL) DAST
Closest papers by meaning
- Nonlinear Color Transfer via Learnable Bezier Flows (CVPR 2026 · similarity 0.89)
- PixelART: Image-to-Layer Decomposition without Latents or Text-to-Image Pretraining (NeurIPS 2026 · similarity 0.83)
- Multi-Modal Synergistic Implicit Image Enhancement for Efficient Optical Flow Estimation (CVPR 2025 · similarity 0.83)
- Image Referenced Sketch Colorization Based on Animation Creation Workflow (CVPR 2025 · similarity 0.83)
- Improving the Training of Rectified Flows (NeurIPS 2024 · similarity 0.83)
- Text-to-Image Rectified Flow as Plug-and-Play Priors (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.