Research map / Vision-language and multimodal / Multimodal LLMs

SynthRGB-T: Language-Vision Guided Image Translation for Diversity Synthesis

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SynthRGB-T, a vision-language guided latent diffusion framework for bidirectional infrared-visible translation with diverse outputs, and builds RGBTSynth-86K to synthesize 50K paired samples.

Problem

Existing infrared-visible translation methods are unidirectional, one-to-one, closed-set, and lack diversity, limiting scalability and generalization for multimodal data synthesis.

Approach

Formulates translation as a vision-language guided denoising diffusion process with a Visual Grounding Pipeline (RAM, Grounding DINO, SAM, CLIP) feeding Q-Former foreground tokens, a decoupled injection mechanism that routes prospect and content guidance to different U-Net resolutions, and a Dual-Conditional Cross-Attention that concatenates keys/values from image and text features.

Key results

• On the I2V task, SynthRGB-T reports LLVIP FID 34.2 and SSIM 0.885 versus CM-Diff's 39.8 and 0.718.
• On the V2I task, SynthRGB-T reports LLVIP LPIPS 0.053 and SSIM 0.922, against the best baseline values of 0.089 and 0.774 (DiffV2IR).
• Ablation: removing DIM, DCCA, or Lcons each degrades results; the full model reaches FID 38.7 vs 144.2 for the baseline with none of the components.

Surprising findings

• Adding the DIM without VGP-based guidance did not improve performance; the authors report that using all cross-attention layers adds cost without gain.

Limitations

• Baselines were independently retrained per benchmark, so comparisons depend on those retrainings.
• Diversity is measured via LPIPS over generated samples, which is a proxy rather than a semantic diversity metric.
• Evaluation is limited to infrared-visible and single-modality benchmarks; medical domain left to future work.

Experimental setup

Datasets: AVMS LLVIP M3FD RoadScene DroneVehicle AVIID MSRS VisDrone DERTAC COCO-2017 CTIR IRSTD-1k

Benchmarks: M3FD LLVIP AVMS CTIR VisDrone

Metrics: NIQE LPIPS FID SSIM

Baselines: CUT CycleGAN StegoGAN UNIT I2V-GAN LG-Diff DiffV2IR CM-Diff

Builds on: Stable Diffusion BLIP-Diffusion DDIM Recognize Anything Grounding DINO Segment Anything

Closest papers by meaning

• Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis (CVPR 2025 · similarity 0.88)
• Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge (NeurIPS 2025 · similarity 0.88)
• MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesis (AAAI 2025 · similarity 0.87)
• LoMo: Local Modality Substitution for Deeper Vision-Language Fusion (NeurIPS 2026 · similarity 0.87)
• CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.87)
• IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting (CVPR 2026 · similarity 0.87)