Research map / Image generation / Image restoration
HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyperFree, a channel-adaptive, tuning-free promptable foundation model for hyperspectral imagery that builds its embedding layer from a learned full-spectrum wavelength weight dictionary, trained on the Hyper-Seg dataset generated by a SAM-based data engine.
Problem
Hyperspectral remote sensing foundation models require per-image fine-tuning because hyperspectral sensors have varying channel counts and wavelengths, which is costly in hardware and time.
Approach
A learnable wavelength weight dictionary (221 keys covering 400-2500 nm at 10 nm) generates the patch embedding kernel dynamically for any input channel set, and prompt-mask-feature interaction uses feature cosine distance to recover multiple semantic masks from a single prompt, enabling tuning-free hyperspectral tasks under visual prompt engineering.
Key results
- In tuning-free mode with one prompt, HyperFree reaches 93.39 OA on LongKou versus 92.65 for MambaHSI (5-shot), and about 12 F1 points gain on HC per the paper.
- On HOCC (HongHu), HyperFree achieves F1 72.52 versus 63.69 for the best specialized baseline PAN (5-shot).
- On change detection (River), zero-shot HyperFree obtains IoU 0.3649 which is ranked 6th, while on Hermiston it reaches IoU 0.5851 (1st).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HyperFree rather than the strongest listed 5-shot classifier | LongKou classification · LongKou | OA: 92.65 -> 93.39 | Helped |
| Use HyperFree rather than the strongest listed 5-shot classifier | HanChuan classification · HanChuan | OA: 73.33 -> 75.47 | Helped |
| Use HyperFree rather than the strongest listed 5-shot method | HongHu one-class classification · HongHu | F1: 63.69 -> 72.52 | Helped |
| Use HyperFree rather than the strongest listed 5-shot method | XiongAn one-class classification · XiongAn | F1: 46.34 -> 52.50 | Helped |
| Use HyperFree rather than the strongest listed method | Airport target detection · Airport | AUC(D,F): 0.9929 -> 0.9806 | Hurt |
| Use HyperFree rather than the strongest listed method | Cri target detection · Cri | AUC(D,F): 0.9987 -> 0.9791 | Hurt |
| Use HyperFree rather than the strongest listed anomaly detector | Beach1 anomaly detection · Beach1 | AUC(D,F): 0.9842 -> 0.9730 | Hurt |
| Use HyperFree rather than the strongest listed anomaly detector | Beach2 anomaly detection · Beach2 | AUC(D,F): 0.9627 -> 0.9566 | Hurt |
| Use HyperFree rather than the strongest listed change detector | River change detection · River | IoU: 0.4534 -> 0.3649 | Hurt |
| Use HyperFree rather than the strongest listed change detector | Hermiston change detection · Hermiston | IoU: 0.5361 -> 0.5851 | Helped |
| remove the βc encoding branch (ablation) | HyperFree · LongKou | OA: 93.40 -> 73.18 | Hurt |
| remove the βk encoding branch (ablation) | HyperFree · LongKou | OA: 93.40 -> 66.63 | Hurt |
Surprising findings
- On the River change detection dataset, zero-shot HyperFree ranks 6th in IoU, below several 5-shot specialized models.
- Applying the same PMF interaction to SAM yields a drop of roughly 10-30 points, indicating that hyperspectral-specific training is essential.
Limitations
- Zero-shot change detection underperforms some specialized models on the River dataset.
- The Hyper-Seg masks are generated automatically by SAM-H from pseudo-color triplets, not from hyperspectral expert labels.
Experimental setup
Datasets: Hyper-Seg fMoW SpaceNet LongKou HanChuan HongHu XiongAn Airport Cri Beach1 Beach2 River
Benchmarks: LongKou HanChuan HongHu XiongAn Airport Cri Beach1 Beach2 River Hermiston
Metrics: OA AA KA F1 precision recall AUC(D,F) AUC_ODP IoU
Baselines: SVM HybridSN FCN FPGA SSFTT MambaHSI OCSVM nnPU BSVM PAN OC Loss T-HOneCls
Builds on: Segment Anything Model HyperSigma MAE
Closest papers by meaning
- LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift (NeurIPS 2026 · similarity 0.86)
- Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning (CVPR 2025 · similarity 0.85)
- Hyperspectral Image Reconstruction via Combinatorial Embedding of Cross-Channel Spatio-Spectral Clues (AAAI 2024 · similarity 0.84)
- S2MAE: A Spatial-Spectral Pretraining Foundation Model for Spectral Remote Sensing Data (CVPR 2024 · similarity 0.84)
- UHR-BAT: Budget-Aware Token Compression Vision-Language model for Ultra-High-Resolution Remote Sensing (ICML 2026 · similarity 0.83)
- Accounting for Spatial Variability with the Histogram of Oriented Gradients Based Masking Improves Performance of Masked Autoencoder over Hyperspectral Satellite Imagery (Student Abstract) (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.