Research map / Image generation / Image restoration
Binarized Neural Network for Multi-spectral Image Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BNNPan, a binarized neural network for pan-sharpening that uses frequency-wise binary wavelet convolutions, full-precision statistic normalization, and a latent diffusion prior to compensate for binarization information loss.
Problem
Deep learning pan-sharpening models are too computationally and memory-heavy for resource-constrained satellites, and naive binarization causes significant information degradation in pixel-level fusion.
Approach
Binarize features after a wavelet decomposition so each frequency subband gets its own binary convolution branch, normalize binary outputs using full-precision residual statistics, and inject a latent diffusion prior (via FiLM-style scale/shift) plus channel attention to recover lost information.
Key results
- On WV3 reduced-resolution, BNNPan reaches PSNR 38.38 dB and SAM 3.1957, outperforming all binary baselines (best other binary BTM 36.96 dB, BBCU 37.08 dB).
- On GF2 reduced-resolution, BNNPan achieves PSNR 41.84 dB versus 39.85 dB for the best binary baseline FABNet.
- Ablation on WV3: removing the full-precision-statistics normalization drops PSNR from 38.38 to 38.01 dB, and removing the latent diffusion prior drops it to 36.89 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DWT-based frequency-wise processing (ablation) | BNNPan · WorldView-3 reduced resolution | PSNR: 38.0056 -> 38.3839 | Helped |
| Add full-precision-statistics normalization to binary convolution (ablation) | BNNPan · WorldView-3 reduced resolution | PSNR: 37.9273 -> 38.3839 | Helped |
| Add Latent Diffusion Prior Compensation (ablation) | BNNPan · WorldView-3 reduced resolution | PSNR: 36.8852 -> 38.3839 | Helped |
| Increase PIBF module count from 2 to 4 (ablation) | BNNPan · WorldView-3 reduced resolution | PSNR: 38.1375 -> 38.3839 | Helped |
| Increase PIBF module count from 4 to 6 (ablation) | BNNPan · WorldView-3 reduced resolution | PSNR: 38.3839 -> 38.4311 | Helped |
| Use BNNPan instead of HFEAN | WorldView-3 reduced-resolution pan-sharpening · WorldView-3 reduced resolution | PSNR: 38.8847 -> 38.3839 | Hurt |
| Use BNNPan instead of FABNet | Gaofen-2 reduced-resolution pan-sharpening · Gaofen-2 reduced resolution | PSNR: 39.8453 -> 41.8447 | Helped |
| Use BNNPan instead of BBCU | QuickBird reduced-resolution pan-sharpening · QuickBird reduced resolution | PSNR: 37.4267 -> 37.9446 | Helped |
Surprising findings
- Binarization applied in the frequency domain preserves more feature information than spatial-domain binarization, with different subbands responding differently.
Limitations
- Full-precision baselines still often outperform BNNPan in several metrics.
- Requires a three-stage training strategy with a separate latent diffusion module, adding training complexity.
- Evaluated only on three remote sensing datasets.
Experimental setup
Datasets: WorldView-3 GaoFen-2 QuickBird
Benchmarks: WorldView-3 GaoFen-2 QuickBird
Metrics: SAM ERGAS Q2n PSNR Dλ Ds HQNR
Baselines: PNN MSDCNN GPPNN FusionNet LAGConv HFEAN HFIN BNN IRNet ReActNet BTM E2FIF
Builds on: ReActNet XNOR-Net BBCU PNN FusionNet
Closest papers by meaning
- Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening (AAAI 2025 · similarity 0.86)
- Progressive High-Frequency Reconstruction for Pan-Sharpening with Implicit Neural Representation (AAAI 2024 · similarity 0.86)
- Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration (AAAI 2024 · similarity 0.86)
- Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening (AAAI 2025 · similarity 0.86)
- Frequency-Adaptive Pan-Sharpening with Mixture of Experts (AAAI 2024 · similarity 0.86)
- FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.