Research map / Architectures and efficiency / Attention and transformers
Occlusion-Embedded Hybrid Transformer for Light Field Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OHT, a hybrid CNN-Transformer for light field super-resolution that embeds photometric-consistency occlusion maps via an occlusion-embedded mix layer and uses SASep-Conv plus angular self-attention.
Problem
Light field super-resolution struggles to efficiently capture both global and local spatial-angular correlations, and existing methods ignore occlusion geometry, causing errors in occluded regions.
Approach
Generates per-view occlusion maps from a disparity estimate and photometric consistency, fuses them with LR features through a 3D-conv mix layer, and combines a lightweight spatial-angular separable convolution with angular-dimension self-attention and a convolutional feed-forward layer in a hybrid transformer.
Key results
- OHT-B reaches 35.66 dB PSNR on EPFL x2 SR and 32.55 dB average PSNR at x4 SR (SSIM 0.9480) with 2.31M parameters.
- OHT-B outperforms DET by 0.08 dB PSNR at x2 SR on average.
- Removing the occlusion-embedded mix layer drops x4 PSNR from 32.55 to 32.30 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the occlusion-embedded mix layer (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.30 -> 32.55 | Helped |
| concatenate the occlusion and light-field features before Nmix instead of adding them (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.45 -> 32.55 | Helped |
| use 3D convolution in Nmix instead of 2D convolution (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.48 -> 32.55 | Helped |
| use SASep-Conv instead of Conv3D (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.41 -> 32.55 | Helped |
| add the spatial branch to SASep-Conv (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.45 -> 32.55 | Helped |
| add the angular branch to SASep-Conv (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.47 -> 32.55 | Helped |
| use ASA instead of channel attention (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.39 -> 32.55 | Helped |
| use ASA instead of Restormer attention (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.42 -> 32.55 | Helped |
| use ASA instead of LFT attention (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.46 -> 32.55 | Helped |
| use ASA instead of EPIT attention (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.49 -> 32.55 | Helped |
| use CFFL instead of FFN (ablation) | OHT-B · BasicLFSR ×4 SR | PSNR (dB): 32.33 -> 32.55 | Helped |
Surprising findings
- On several datasets (e.g., HCInew x2 and some x4 cases), DET or EPIT match or exceed OHT, so OHT is not uniformly best.
Limitations
- Model complexity is not yet optimized, as stated by the authors.
- OHT does not outperform all baselines on every test set (e.g., DET is better on some cases).
Experimental setup
Datasets: HCI-new HCI-old EPFL INRIA STFGantry
Benchmarks: EPFL HCInew HCIold INRIA STFgantry
Metrics: PSNR SSIM parameters
Baselines: EPIT DET LFT DPT DistgSSR LF-IINet DFNet LF-InterNet ResLF LFSSR VDSR EDSR
Builds on: EPIT OACCNet Restormer
Closest papers by meaning
- Learning Light Field Angular Super-Resolution via a Geometry-Aware Network (AAAI 2020 · similarity 0.87)
- From Pixels to Views: Learning Angular-Aware and Physics-Consistent Representations for Light Field Microscopy (NeurIPS 2025 · similarity 0.84)
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.84)
- Fast Omni-Directional Image Super-Resolution: Adapting the Implicit Image Function with Pixel and Semantic-Wise Spherical Geometric Priors (AAAI 2025 · similarity 0.84)
- CFAT: Unleashing Triangular Windows for Image Super-resolution (CVPR 2024 · similarity 0.84)
- OSFFNet: Omni-Stage Feature Fusion Network for Lightweight Image Super-Resolution (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.