Research map / Image generation / Super-resolution
FIPER: Factorized Features for Robust Image Super-Resolution and Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Factorized Features, a basis-coefficient image representation with sawtooth coordinate transform and multi-frequency modulation, applied to plug into SR backbones (SwinIR/HAT/DAT/ATD) and a learned image compression pipeline (TCM) with multi-image basis merging.
Problem
Existing SR and learned image compression methods focus on network architecture and under-model fine details and repetitive structures; a more expressive image representation is needed for both tasks.
Approach
Represent images as learned coefficient maps times learned basis maps, with spatially sawtooth-transformed sampling of bases and multi-frequency modulation (sin/cos at several alpha scales), replacing the simple feature-map output of networks; for compression, a shared merged basis is transmitted across multiple images.
Key results
- HAT-L-F achieves 33.75 dB PSNR on Set5 and 29.87 dB on Set14 at 4x SR, surpassing HAT-L and the other listed SR baselines.
- TCM-HAT-L-F achieves -21.09% BD-Rate vs VTM, the best among listed learned codecs (LALIC -15.26%).
- Multi-image variant TCM-HAT-F-multi reaches -20.97% BD-Rate at M=24 versus 27.96% for M=1 on the same pipeline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Factorized Features to HAT-L | HAT-L super-resolution · Urban100 | PSNR: 28.60 -> 29.51 | Helped |
| Reverse the coefficient-first pipeline to basis-first (ablation) | HAT-F super-resolution · Set5 | PSNR: 33.53 -> 33.33 | Hurt |
| Concatenate basis and coefficient instead of applying Factorized Features reconstruction (ablation) | HAT-F super-resolution · Set5 | PSNR: 33.53 -> 33.46 | Hurt |
| Use factor fields instead of full Factorized Features (ablation) | HAT-F-Small super-resolution · Set5 | PSNR: 33.24 -> 33.06 | Hurt |
| Use only sine modulation (ablation) | HAT-F-Small super-resolution · Set5 | PSNR: 33.24 -> 33.16 | Hurt |
| Reduce frequency scalars to {1, 16} (ablation) | HAT-F-Small super-resolution · Set5 | PSNR: 33.24 -> 33.18 | Hurt |
| Train the M=24 basis-merging model from scratch (ablation) | TCM-HAT-F-multi M=24 · VTM anchor (Kodak, Tecnick, CLIC) | BD-Rate (%): -20.97 -> -18.34 | Hurt |
| Freeze the coefficient branch (ablation) | TCM-HAT-F-multi M=24 · VTM anchor (Kodak, Tecnick, CLIC) | BD-Rate (%): -20.97 -> -19.18 | Hurt |
| Increase the number of jointly compressed images from 1 to 24 (ablation) | TCM-HAT-F-multi · VTM anchor (Kodak, Tecnick, CLIC) | BD-Rate (%): 27.96 -> -20.97 | Helped |
| Add the original HAT SR module to TCM instead of the Factorized Features SR module (ablation) | TCM image compression · Kodak λ=0.0025 | PSNR: 30.0834 -> 30.1843 | Helped |
| Use the Factorized Features SR module rather than a randomly initialized SR module (ablation) | TCM image compression · Kodak λ=0.0025 | PSNR: 30.0857 -> 30.4012 | Helped |
Surprising findings
- Reversing the order (basis first, then coefficients) causes a large performance drop, per the authors.
- Single-image basis merging setting (M=1) yields BD-rate of +27.96% versus VTM, worse than the non-merged TCM-HAT-L-F.
Limitations
- Requires further optimization for computation-limited scenarios such as real-time decoding.
- Incorporating semantic information is left for future work.
- Per-image compression of the merged basis adds latency at small M (M=1 gives BD-rate worse than baseline).
Experimental setup
Datasets: ImageNet DIV2K Flickr2K REDS Vimeo-90K
Benchmarks: Set5 Set14 B100 Urban100 Manga109 Kodak CLIC Tecnick REDS4 Vimeo-90K-T Vid4
Metrics: PSNR SSIM BD-Rate inference time params MACs
Baselines: EDSR RCAN SwinIR ATD DAT RGT PFT HAT VTM TCM LALIC ELIC
Builds on: Factor Fields Fourier Features HAT TCM SwinIR
Closest papers by meaning
- Auto-Encoded Supervision for Perceptual Image Super-Resolution (CVPR 2025 · similarity 0.86)
- GaussianSR: High Fidelity 2D Gaussian Splatting for Arbitrary-Scale Image Super-Resolution (AAAI 2025 · similarity 0.85)
- Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss (CVPR 2024 · similarity 0.85)
- AdaFormer: Efficient Transformer with Adaptive Token Sparsification for Image Super-resolution (AAAI 2024 · similarity 0.85)
- Lightweight Image Super-Resolution via Flexible Meta Pruning (ICML 2024 · similarity 0.85)
- Learning Coupled Dictionaries from Unpaired Data for Image Super-Resolution (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.