Research map / Learning theory and algorithms / Prediction and decision losses

Spectral-Spatial Interpretation

NeurIPS 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Provable post-hoc explanations

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Spectral-Spatial Interpretation (SSI), which computes a full position-channel attribution matrix by applying the Shapley value over channels at each grid position within the observed-context conditional game.

Problem

Post-hoc explanations either give spatial heatmaps or global channel rankings, missing spatially localized channel relevance in multispectral images.

Approach

For each spatial position, fix the observed surrounding context and apply the standard Shapley value only over that position's channels (a conditional position game), yielding a full position-channel attribution matrix whose channel sums equal the position-level contextual contrast; contrasts with Owen value, which averages over external spatial coalitions.

Key results

• On HLS Burn Scar, SSI achieves joint deletion AUC .155 vs .173 for MC-Owen while using about 6% of its forward-pass budget (4,480 vs 76,800).
• SSI achieves the best joint deletion AUC and highest insertion AUC on all three datasets under both mean and zero baselines (e.g., EuroSAT joint deletion .1397 vs .2078 for DeepSHAP under mean baseline).
• On HLS Burn Scar Top-10% spatial mIoU reaches .389 for SSI vs .277 for DeepSHAP and .191 for RISE.

Surprising findings

• A rank-1 spatial-times-channel factorization (RISE-based control) substantially worsens deletion, indicating channel relevance is position-dependent.

Limitations

• Evaluated only on three datasets with small channel counts (C=3, 6, 13); scalability to hyperspectral data untested.
• Forward-pass counts are analytical, not measured wall-clock times; gradient-based costs not directly comparable.
• Mean/zero baseline semantics of the conditional game depend on masking choice.

Experimental setup

Datasets: CUB-200-2011 EuroSAT HLS Burn Scar

Benchmarks: CUB-200-2011 EuroSAT HLS Burn Scar

Metrics: joint deletion AUC rank-1 deletion AUC grid-sum insertion AUC channel deletion AUC spatial mIoU spectral hit rate ROAD LeRF/MoRF AUC

Baselines: Integrated Gradients DeepSHAP KernelSHAP RISE MC-Owen Expected Gradients Guided IG FullGrad

Builds on: Shapley value Owen value RISE KernelSHAP

Closest papers by meaning

• Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion (ICLR 2025 · similarity 0.84)
• Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.84)
• SpatialViz-Bench: A Cognitively-Grounded Benchmark for Diagnosing Spatial Visualization in MLLMs (ICLR 2026 · similarity 0.83)
• MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence (ICLR 2026 · similarity 0.83)
• Spatial Reasoning with Denoising Models (ICML 2025 · similarity 0.83)
• Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-Language Models (ICLR 2026 · similarity 0.83)