Research map / Architectures and efficiency / Attention and transformers
Hierarchical Neural Operator Transformer with Learnable Frequency-aware Loss Prior for Arbitrary-scale Super-resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HiNOTE, a hierarchical Galerkin-type transformer neural operator with sinc-filter up/downsampling and a learnable frequency-aware loss prior for arbitrary-scale super-resolution of scientific data.
Problem
Arbitrary-scale super-resolution of scientific data (turbulence, weather, MRI, radar) where MLP-based implicit representations suffer spectral bias and per-pixel L1/L2 losses blur high-frequency details.
Approach
Replace the MLP inference head of implicit neural representations with a hierarchical neural operator using Galerkin-type linear self-attention and sinc-filtered up/downsampling for alias-aware representation, plus a model-agnostic loss weighting derived from the difference between predictions and spectrally resized inputs.
Key results
- At x4.6 arbitrary-scale SR on turbulence, HiNOTE reports PSNR 51.225 vs SRNO 50.081 and LTE 50.985.
- At x8.2 on turbulence, HiNOTE reaches PSNR 40.002 vs SRNO 37.852, and the paper reports an average 20.11% improvement over SRNO across ratios.
- Loss prior ablation improves MSE by 1.32% (turbulence) to 21.78% (kinetic energy) at x4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HiNOTE instead of the strongest listed baseline | arbitrary-scale SR models · Turbulence ×4.6 | PSNR: 50.985 -> 51.225 | Helped |
| use HiNOTE instead of the strongest listed baseline | arbitrary-scale SR models · Turbulence ×8.2 | PSNR: 37.852 -> 40.002 | Helped |
| use HiNOTE instead of the strongest listed baseline | arbitrary-scale SR models · Turbulence ×15.7 | PSNR: 30.759 -> 32.164 | Helped |
| use HiNOTE instead of the strongest listed baseline | arbitrary-scale SR models · Turbulence ×32 | PSNR: 26.708 -> 26.772 | Helped |
| increase feature upsampling ratio from ×1 to ×4 (ablation) | HiNOTE · test data at upsampling ratio 8 | MSE: 9.869e-5 -> 8.919e-5 | Helped |
| increase feature upsampling ratio from ×2 to ×4 (ablation) | HiNOTE · test data at upsampling ratio 8 | MSE: 9.307e-5 -> 8.919e-5 | Helped |
| replace Galerkin attention with Vanilla attention (ablation) | HiNOTE · Turbulence 128 ×128 | MSE: 5.201e-6 -> 3.382e-5 | Hurt |
| replace Galerkin attention with FAVOR (ablation) | HiNOTE · Turbulence 128 ×128 | MSE: 5.201e-6 -> 1.002e-5 | Hurt |
| replace Galerkin attention with ProbSparse (ablation) | HiNOTE · Turbulence 128 ×128 | MSE: 5.201e-6 -> 9.941e-6 | Hurt |
| add frequency-aware loss prior (ablation) | HiNOTE · Turbulence ×4 | MSE: 6.194e-6 -> 6.112e-6 | Helped |
| add frequency-aware loss prior (ablation) | HiNOTE · Temperature ×4 | MSE: 2.061e-5 -> 1.983e-5 | Helped |
| add frequency-aware loss prior (ablation) | HiNOTE · Kinetic Energy ×4 | MSE: 1.455e-4 -> 1.138e-4 | Helped |
Surprising findings
- Bilinear interpolation often outperforms deep learning models at the x32 upsampling ratio, indicating limited extrapolation beyond training ratios up to 4.
- Galerkin attention outperformed vanilla, FAVOR and ProbSparse attention in the ablation, with lower FLOPs and time.
Limitations
- Extrapolation beyond training upsampling ratio (max 4) degrades, with bilinear interpolation winning at x32.
- Training limited to crops due to memory; experiments on 4 datasets only with a single GPU setup.
- Loss prior is described with multiple implementation variants (focal-style and two-stage) with details deferred to appendix.
Experimental setup
Datasets: Kraichnan turbulence (simulated) ERA5 global weather SEVIR MRI brain scans
Benchmarks: Kraichnan turbulence ERA5 weather (temperature, kinetic energy, water vapor) SEVIR MRI
Metrics: MSE PSNR SSIM
Baselines: SRCNN ESPCN EDSR WDSR SwinIR MetaSR LIIF LTE DFNO SRNO Bilinear Bicubic
Builds on: Galerkin transformer neural operator LIIF LTE SRNO
Closest papers by meaning
- DiffFNO: Diffusion Fourier Neural Operator (CVPR 2025 · similarity 0.88)
- HIIF: Hierarchical Encoding based Implicit Image Function for Continuous Super-resolution (CVPR 2025 · similarity 0.86)
- Exploring the Low-Pass Filtering Behavior in Image Super-Resolution (ICML 2024 · similarity 0.86)
- Implicit Representations via Operator Learning (ICML 2024 · similarity 0.85)
- Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains (ICML 2024 · similarity 0.85)
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.