Research map / Image generation / Image restoration
OTIAS: OcTree Implicit Adaptive Sampling for Multispectral and Hyperspectral Image Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OTIAS, an octree-based implicit neural representation with adaptive synthesis kernels for multispectral and hyperspectral image fusion, reporting state-of-the-art results on CAVE and Harvard at 4x scaling.
Problem
Multispectral and hyperspectral image fusion (MHIF) fuses a low-resolution hyperspectral image with a high-resolution multispectral image; prior INR-based fusion methods use a quadtree spatial sampling and share one kernel across all channels, ignoring channel-wise and spectral structure.
Approach
Extends the quadtree INR sampling into an octree hierarchy that splits channels into spatial positions (pixel-shuffle style) so sampling spans spatial and spectral dimensions, and generates channel-specific depthwise interpolation kernels per coordinate via a lightweight adaptive synthesis kernel (ASK) module.
Key results
- On CAVE at 4x, OTIAS reaches 52.43 dB PSNR, 1.94 SAM, 0.99 ERGAS, versus 52.22 dB for QIS (second best).
- On Harvard at 4x, OTIAS achieves 49.08 dB PSNR, 0.2 dB above the second-best method, with 50.7% fewer parameters than DSPNet.
- Adding ASK to JIIF and QIS improves PSNR by 0.29 dB and 0.10 dB respectively with negligible parameter overhead.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OTIAS instead of QIS | CAVE ×4 fusion · CAVE ×4 | PSNR: 52.22 -> 52.43 | Helped |
| use OTIAS instead of QIS | Harvard ×4 fusion · Harvard ×4 | PSNR: 48.88 -> 49.08 | Helped |
| add ASK (ablation) | JIIF on CAVE · CAVE ×4 | PSNR: 50.93 -> 51.22 | Helped |
| add ASK (ablation) | JIIF on CAVE · CAVE ×4 | SAM: 2.13 -> 1.98 | Helped |
| add ASK (ablation) | JIIF on CAVE · CAVE ×4 | ERGAS: 1.12 -> 1.02 | Helped |
| add ASK (ablation) | QIS on CAVE · CAVE ×4 | PSNR: 52.22 -> 52.32 | Helped |
| add ASK (ablation) | QIS on CAVE · CAVE ×4 | SAM: 1.98 -> 1.97 | Helped |
| add ASK (ablation) | QIS on CAVE · CAVE ×4 | ERGAS: 1.02 -> 0.99 | Helped |
| increase OTH from 1 to 3 layers (ablation) | OTIAS on CAVE · CAVE ×8 | SAM: 2.59 -> 2.51 | Helped |
| increase OTH from 1 to 3 layers (ablation) | OTIAS on CAVE · CAVE ×8 | ERGAS: 0.74 -> 0.70 | Helped |
| increase OTH from 1 to 3 layers (ablation) | OTIAS on CAVE · CAVE ×8 | SSIM: 0.996 -> 0.996 | No clear effect |
Surprising findings
- OTIAS has slightly worse SAM and SSIM than DSPNet on Harvard despite higher PSNR.
- Increasing octree layers improves SAM at 8x but adds only marginal parameters.
Limitations
- Slightly inferior SAM and SSIM to DSPNet on Harvard at 4x.
- Higher FLOPs (8.722G) than some baselines such as SSRNet and HSRNet.
- Evaluated only on two simulated datasets (CAVE and Harvard) with simulated LR-HSI/HR-MSI pairs.
Experimental setup
Datasets: CAVE Harvard
Benchmarks: CAVE Harvard
Metrics: PSNR SAM ERGAS SSIM
Baselines: CSTF-FUS LTTR LTMR IR-TenSR SSRNet ResTFNet HSRNet MoG-DCN Fusformer DHIF PSRT 3DT-Net
Builds on: LIIF JIIF QIS PixelShuffle
Closest papers by meaning
- Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion (NeurIPS 2024 · similarity 0.86)
- Unmixing Before Fusion: A Generalized Paradigm for Multi-Source-based Hyperspectral Image Synthesis (CVPR 2024 · similarity 0.85)
- OTPNet: ODE-inspired Tuning-free Proximal Network for Remote Sensing Image Fusion (AAAI 2025 · similarity 0.84)
- Hyperspectral Pansharpening via Diffusion Models with Iteratively Zero-Shot Guidance (CVPR 2025 · similarity 0.83)
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.83)
- A Selective Re-learning Mechanism for Hyperspectral Fusion Imaging (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.