Research map / Image generation / Image restoration
High-Resolution Image Harmonization with Adaptive-Interval Color Transformation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AICT, a lightweight transformer-based method that predicts position-dependent 3D LUTs at low resolution and uses adaptive sampling intervals plus a global weight module for pixel-wise color transformation in high-resolution image harmonization.
Problem
High-resolution image harmonization methods rely on global color adjustments or upsampled parameter maps, which ignore local variations and cause inharmonious results at full resolution.
Approach
Predict pixel-wise color transformations as position-dependent 3D LUTs from low-resolution inputs, cascade two such LUTs so the first redistributes color values to adaptively set sampling intervals for the second, and add a global image-level weight learning module to model overall brightness/color shifts from background.
Key results
- On full-resolution iHarmony4, AICT reaches fMSE 228.43 and PSNR 39.40, versus PCT-Net's 238.27 and 39.28.
- On HAdobe5k at 2048x2048, AICT achieves fMSE 147.99 versus PCT-Net's 156.56.
- On ccHarmony at 256x256, AICT achieves fMSE 232.66 and PSNR 38.46, the best among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AICT instead of PCT-Net | iHarmony4 · iHarmony4 full-resolution, All | fMSE: 238.27 -> 228.43 | Helped |
| use AICT instead of PCT-Net | iHarmony4 · iHarmony4 full-resolution, All | SSIM: 0.9911 -> 0.9910 | No clear effect |
| use AICT instead of PCT-Net | iHarmony4 · iHarmony4 256 × 256, All | fMSE: 216.25 -> 204.67 | Helped |
| use AICT instead of CDTNet-256 | HAdobe5k · HAdobe5k 1024 × 1024 | fMSE: 152.13 -> 156.81 | Hurt |
| remove adaptive interval learning (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 239.34 | Hurt |
| use 2 LUTs for adaptive interval learning (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 241.79 | Hurt |
| use 3 LUTs for adaptive interval learning (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 231.91 | Hurt |
| use multiple LUTs for color transformation (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 456.98; 228.43 -> 487.60 | Hurt |
| use global RGB-to-RGB transformations instead of pixel-wise transformations (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 357.61 | Hurt |
| remove global consistent weight learning module (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 247.48 | Hurt |
| predict spatially varying weight parameters (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 230.24 | Hurt |
| set number of knots K to 6 (ablation) | AICT · iHarmony4 full-resolution, All | fMSE: 228.43 -> 241.67 | Hurt |
Surprising findings
- Using more cascaded LUTs for color transformation (Tra x2, Tra x3) or alternating cascades substantially degrades performance (fMSE rises to over 440), attributed to training difficulty.
- Predicting spatially varying weights for the global weight module does not improve performance over image-level weights.
Limitations
- Fails on foreground elements such as mirrors and glass, since the model cannot simulate light-material interaction.
- Lower performance on the HDay2night subset, attributed to its small number of training images.
Experimental setup
Datasets: iHarmony4 ccHarmony HAdobe5k real composite dataset (Cong et al. / CDTNet)
Benchmarks: iHarmony4 full-resolution iHarmony4 256x256 HAdobe5k 1024/2048 resolution HAdobe5k over4K/over5K ccHarmony 256x256 real composite dataset
Metrics: fMSE MSE PSNR SSIM FLOPs memory inference time user study score
Baselines: AdaInt SepLUT Harmonizer DCCF PCT-Net CDTNet S2CRNet iS2AM GiftNet
Builds on: AdaInt SepLUT PCT-Net DoveNet
Closest papers by meaning
- Progressive Painterly Image Harmonization from Low-Level Styles to High-Level Styles (AAAI 2024 · similarity 0.86)
- Real-Time Exposure Correction via Collaborative Transformations and Adaptive Sampling (CVPR 2024 · similarity 0.84)
- FRIH: Fine-Grained Region-Aware Image Harmonization (AAAI 2024 · similarity 0.84)
- Painterly Image Harmonization by Learning from Painterly Objects (AAAI 2024 · similarity 0.82)
- GLUT: 3D Gaussian Lookup Table for Continuous Color Transformation (NeurIPS 2026 · similarity 0.82)
- Color Shift Estimation-and-Correction for Image Enhancement (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.