Research map / Image generation / Editing and style transfer
ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ThermalGen, a flow-based (SiT) latent generative model for RGB-to-thermal translation with RGB-latent conditioning and learnable dataset-specific style embeddings, trained on eight public plus three new satellite-aerial RGB-T datasets.
Problem
Synthesizing thermal images from RGB to scale paired RGB-thermal data for cross-modal tasks, where existing translation methods generalize poorly across sensors, viewpoints, and environments.
Approach
A latent flow-matching transformer (SiT) conditioned on an RGB KL-VAE latent (via concatenation or cross-attention) and on learnable dataset-specific style embeddings injected through adaLN-Zero, with classifier-free guidance over styles, so one model covers many RGB-T styles and new styles can be added by appending embeddings.
Key results
- On Bosonplus-day, ThermalGen-L/2 reaches FID 76.91 and LPIPS 0.35, best among compared methods, versus 137.68 for the best baseline BBDM in FID.
- On NII-CU, ThermalGen-L/2 reaches PSNR 26.44 and FID 69.30 versus 19.46 PSNR and 118.60 FID for the best GAN baseline pix2pixHD.
- On LLVIP, DiffV2IR (trained on the LLVIP test set) outperforms ThermalGen, and on Boson-night and FLIR ThermalGen underperforms; CFG scale 8.0 reduces Boson-night FID from 161.22 to 116.46.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · Bosonplus-day | FID: 157.65 -> 76.91 | Helped |
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · Bosonplus-night | FID: 72.02 -> 75.80 | Hurt |
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · LLVIP | FID: 50.10 -> 238.60 | Hurt |
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · NII-CU | FID: 118.60 -> 69.30 | Helped |
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · AVIID | FID: 51.61 -> 29.05 | Helped |
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · M³FD | FID: 37.74 -> 35.82 | Helped |
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · MSRS | FID: 61.52 -> 52.31 | Helped |
| Use ThermalGen-L/2 instead of the strongest listed baseline | RGB-to-thermal translation · FLIR | FID: 37.83 -> 70.09 | Hurt |
| Set CFG scale to 8.0 (ablation) | ThermalGen-L/2 · Boson-night | FID: 161.22 -> 116.46 | Helped |
| Set CFG factor to 4.0 (ablation) | ThermalGen-L/2 · FLIR | FID: 70.09 to 63.43 | Helped |
| Use fixed style embeddings instead of CLIP embeddings (ablation) | ThermalGen · Bosonplus-day | FID: 106.08 -> 76.91 | Helped |
| Use fixed style embeddings instead of CLIP embeddings (ablation) | ThermalGen · Bosonplus-night | FID: 89.65 -> 75.80 | Helped |
Surprising findings
- DDIM trained on the curated data often produces random samples resembling training thermal images rather than conditioning on the RGB input.
- CLIP-based embeddings from paired RGB images performed worse than fixed learnable style embeddings (e.g., Bosonplus-day FID 106.08 vs 76.91).
Limitations
- Underperforms DiffV2IR and other baselines on Boson-night, LLVIP, and FLIR, attributed to low-contrast thermal imagery, distribution shift, and extreme lighting.
- Reports no error bars or statistical significance due to compute constraints.
- Bosonplus data release restricted to 2 of 3 regions for compliance.
Experimental setup
Datasets: Boson-night CalTech LLVIP NII-CU AVIID M3FD Freiburg SMOD MSRS KAIST FLIR DJI-day
Benchmarks: Boson-night Bosonplus-day Bosonplus-night LLVIP NII-CU AVIID M3FD MSRS FLIR
Metrics: PSNR SSIM FID LPIPS
Baselines: pix2pix CycleGAN pix2pixHD VQGAN DDIM BBDM DiffV2IR
Builds on: SiT latent diffusion adaLN-Zero DiffV2IR pix2pix
Closest papers by meaning
- ThermalGaussian: Thermal 3D Gaussian Splatting (ICLR 2025 · similarity 0.87)
- Enhancing Infrared Vision: Progressive Prompt Fusion Network and Benchmark (NeurIPS 2025 · similarity 0.85)
- MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation (NeurIPS 2024 · similarity 0.84)
- SynthRGB-T: Language-Vision Guided Image Translation for Diversity Synthesis (CVPR 2026 · similarity 0.84)
- RGB-to-Polarization Estimation: A New Task and Benchmark Study (NeurIPS 2025 · similarity 0.84)
- Thermal-Aware Low-Light Image Enhancement: A Real-World Benchmark and a New Light-Weight Model (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.