Research map / Image generation / Super-resolution
Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PiSA-SR, a one-step diffusion super-resolution method that trains two LoRA modules on Stable Diffusion (pixel-level with l2 loss, semantic-level with LPIPS and CSD loss) and exposes two guidance scales to adjust fidelity versus perceptual detail at inference.
Problem
Diffusion-prior real-world super-resolution entangles pixel-level fidelity and semantic-level perceptual objectives, so methods struggle to balance them, and users need a way to tune the trade-off at inference without retraining.
Approach
Formulate SD-based SR as learning a residual between LQ and HQ latents, decouple training into a pixel-level LoRA (l2 loss) and a semantic-level LoRA (LPIPS + CSD loss) stacked on the frozen pixel LoRA, and combine them with two guidance scales in a CFG-like formula.
Key results
- Default one-step PiSA-SR reaches LPIPS 0.2672 and MUSIQ 70.15 on RealSR, outperforming other diffusion SR methods in reference and no-reference metrics.
- On RealSR, varying pixel guidance scale to 0.5 gives PSNR 26.75 (peak), while semantic scale 0.8 gives best LPIPS 0.2465 at pix=1.0 with the semantic sweep.
- PiSA-SR default inference takes 0.09 s per 128x128 image on an A100 with 1.30B parameters, faster than OSEDiff (0.12 s) and multi-step methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dual-LoRA PiSA-SR instead of OSEDiff-S1 | SD-based real-world super-resolution · DIV2K | LPIPS: 0.2941 -> 0.2823 | Helped |
| Use dual-LoRA PiSA-SR instead of SeeSR-S50 | SD-based real-world super-resolution · DIV2K | CLIPIQA: 0.6857 -> 0.6927 | Helped |
| Use dual-LoRA PiSA-SR instead of OSEDiff-S1 | SD-based real-world super-resolution · RealSR | LPIPS: 0.2921 -> 0.2672 | Helped |
| Use dual-LoRA PiSA-SR instead of ResShift-S15 | SD-based real-world super-resolution · RealSR | PSNR: 26.31 -> 25.50 | Hurt |
| Use dual-LoRA PiSA-SR instead of OSEDiff-S1 | SD-based real-world super-resolution · DrealSR | LPIPS: 0.2968 -> 0.2960 | Helped |
| Use dual-LoRA PiSA-SR instead of ResShift-S15 | SD-based real-world super-resolution · DrealSR | PSNR: 28.45 -> 28.31 | Hurt |
| Decrease λpix from 1.0 to 0.5 while keeping λsem=1.0 (ablation) | PiSA-SR · RealSR | PSNR: 25.50 -> 26.75 | Helped |
| Decrease λpix from 1.0 to 0.8 while keeping λsem=1.0 (ablation) | PiSA-SR · RealSR | LPIPS: 0.2672 -> 0.2612 | Helped |
| Increase λpix from 1.0 to 1.2 while keeping λsem=1.0 (ablation) | PiSA-SR · RealSR | PSNR: 25.50 -> 24.76 | Hurt |
| Decrease λsem from 1.0 to 0.8 while keeping λpix=1.0 (ablation) | PiSA-SR · RealSR | LPIPS: 0.2672 -> 0.2465 | Helped |
| Increase λsem from 1.0 to 1.2 while keeping λpix=1.0 (ablation) | PiSA-SR · RealSR | CLIPIQA: 0.6702 -> 0.7015 | Helped |
| Increase λsem from 1.0 to 1.2 while keeping λpix=1.0 (ablation) | PiSA-SR · RealSR | PSNR: 25.50 -> 24.59 | Hurt |
Surprising findings
- Pixel-level guidance scale above 1 eventually lowers PSNR and LPIPS even while no-reference metrics keep improving, showing a fidelity-perception trade-off within the adjustable range.
Limitations
- Single pixel-level LoRA may be insufficient for heavy or diverse degradations.
- Adjustable setting incurs a slightly longer inference time than the default one-step version.
Experimental setup
Datasets: LSDIR FFHQ DIV2K RealSR DrealSR
Benchmarks: DIV2K synthetic test (3000 crops) RealSR DrealSR
Metrics: PSNR SSIM LPIPS DISTS FID NIQE CLIPIQA MUSIQ MANIQA
Baselines: StableSR ResShift DiffBIR PASD SeeSR SinSR OSEDiff RealESRGAN BSRGAN LDL
Builds on: OSEDiff Stable Diffusion 2.1 LoRA classifier score distillation LPIPS
Closest papers by meaning
- SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning (NeurIPS 2025 · similarity 0.90)
- SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (CVPR 2024 · similarity 0.89)
- When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution (NeurIPS 2026 · similarity 0.89)
- One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution (NeurIPS 2025 · similarity 0.89)
- Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator (NeurIPS 2025 · similarity 0.88)
- DREAM: Diffusion Rectification and Estimation-Adaptive Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.