Research map / Image generation / Autoregressive and latent generation
Dual-Granularity Semantic Guided Sparse Routing Diffusion Model for General Pansharpening
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SGDiff, a diffusion model for general pansharpening whose sparse mixture-of-experts blocks are routed by scene-level and region-level text semantics from GeoChat, trained jointly on multiple satellite datasets.
Problem
Pansharpening models are sensor-dependent and scene-dependent, degrading on unseen satellites or scene types when trained on imbalanced data.
Approach
Use a geoscience VLM (GeoChat) to produce scene-category and grounding-description text, encode them with CLIP, and use them as dual-granularity gating signals that sparsely activate channel experts inside a 3D transformer diffusion denoiser (SG-MoE).
Key results
- On WV-3 reduced resolution, SGDiff reaches Q8 0.9227, SAM 2.7708, ERGAS 2.0442, outperforming TMDiff (Q8 0.9210, SAM 2.7971).
- On WV-2 zero-shot, SGDiff achieves Q8 0.8577 and ERGAS 3.8683, better than TMDiff (Q8 0.8310, ERGAS 4.1420).
- On WV-3 full resolution, SGDiff reports HQNR 0.9609 versus 0.9553 for TMDiff.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SG-MoE in place of 3D Dynamic Gated FFN (ablation) | SGDiff · WorldView-3 and QuickBird | Q8: 0.9190 -> 0.9227; 0.9344 -> 0.9384 | Helped |
| Add fine-grained grounding descriptions (ablation) | SGDiff · WorldView-3 and QuickBird | Q8: 0.9155 -> 0.9227; 0.9346 -> 0.9384 | Helped |
| Add shared expert isolation (ablation) | SGDiff · WorldView-3 and QuickBird | Q8: 0.9160 -> 0.9227; 0.9326 -> 0.9384 | Helped |
| Add sparse regularization (ablation) | SGDiff · WorldView-3 and QuickBird | Q8: 0.9203 -> 0.9227; 0.9370 -> 0.9384 | Helped |
| Use SGDiff instead of TMDiff | pansharpening model · WorldView-3 reduced resolution | Q8: 0.9210±0.0823 -> 0.9227±0.0821 | Helped |
| Use SGDiff instead of TMDiff | pansharpening model · QuickBird full resolution | HQNR: 0.9075±0.0102 -> 0.9341±0.0105 | Helped |
| Train and evaluate SGDiff on unseen WorldView-2 | SGDiff · WorldView-2 zero-shot | Q8: 0.8310 -> 0.8577 | Helped |
Limitations
- Results on GF-2 are only in the supplementary material, not in the main text.
- Scene categories are a predefined closed set of seven classes for prompting GeoChat.
- Ablation gains over the single-granularity and no-SG-MoE variants are small on some metrics.
Experimental setup
Datasets: PanCollection (WorldView-3, QuickBird, GaoFen-2) WorldView-2 (zero-shot)
Benchmarks: PanCollection WV-3 reduced and full resolution PanCollection QuickBird reduced and full resolution WorldView-2 zero-shot
Metrics: Q8 SAM ERGAS SCC Dλ Ds HQNR
Baselines: C-GSA MTF LGPConv BIMPAN U2Net CrossDiff TMDiff B-TH BDSD-PC SR-D
Builds on: TMDiff DeepMoE Restormer GeoChat DDPM
Closest papers by meaning
- Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening (CVPR 2024 · similarity 0.87)
- PanAdapter: Two-Stage Fine-Tuning with Spatial-Spectral Priors Injecting for Pansharpening (AAAI 2025 · similarity 0.86)
- A General Adaptive Dual-level Weighting Mechanism for Remote Sensing Pansharpening (CVPR 2025 · similarity 0.86)
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.86)
- Pan-LUT: Efficient Pan-sharpening via Learnable Look-Up Tables (NeurIPS 2025 · similarity 0.85)
- SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.