Research map / Detection and segmentation / CNN features
Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SR4IR, which jointly guides super-resolution and a task network via a task-driven perceptual loss computed in the task network's feature space, with cross-quality patch mix augmentation and alternate training, improving recognition on low-resolution images.
Problem
Image recognition tasks (segmentation, detection, classification) degrade on low-resolution inputs, and standard super-resolution does not reliably restore task-relevant high-frequency details.
Approach
A task-driven perceptual loss that measures SR-vs-HR similarity in the feature space of the task network itself (instead of a fixed ImageNet VGG), combined with CQMix patch mixing of SR and HR patches to prevent shortcut learning and an alternate training schedule that alternates between updating the SR network and the task network.
Key results
- On PASCAL VOC segmentation at x4 with SwinIR, SR4IR reaches 62.2 mIoU versus 58.6 for bilinear-upsampled input (ILR→T) and 63.3 for the HR oracle.
- On x8 Stanford Cars classification with SwinIR, SR4IR improves Top-1 accuracy by +12.9% over bilinear-upsampled input (71.4 vs 61.0... see table: 73.9 vs 61.0).
- Replacing the task-driven perceptual loss with a fixed ImageNet VGG perceptual loss gives 53.6 mIoU, and joint on-training perceptual loss collapses to 3.5 mIoU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · PASCAL VOC2012 x4 | mIoU: 61.4 -> 62.2 | Helped |
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · PASCAL VOC2012 x8 | mIoU: 53.9 -> 56.5 | Helped |
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · PASCAL VOC2012 x4 | mAP: 31.5 -> 33.9 | Helped |
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · PASCAL VOC2012 x8 | mAP: 23.7 -> 27.8 | Helped |
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · Stanford Cars x4 | Top-1 accuracy: 84.0 -> 85.1 | Helped |
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · Stanford Cars x8 | Top-1 accuracy: 71.4 -> 73.9 | Helped |
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · CUB-200-2011 x4 | Top-1 accuracy: 72.9 -> 74.7 | Helped |
| Use SR4IR with SwinIR instead of the strongest reported SR-combined baseline | SwinIR · CUB-200-2011 x8 | Top-1 accuracy: 61.2 -> 65.3 | Helped |
| Add TDP loss (ablation) | SR4IR · PASCAL VOC2012 x8 | mIoU: 53.3 -> 55.0 | Helped |
| Add TDP loss (ablation) | SR4IR · Stanford Cars x8 | Top-1 accuracy: 69.9 -> 71.4 | Helped |
| Add CQMix-augmented images to task-network training (ablation) | SR4IR · PASCAL VOC2012 x8 | mIoU: 54.7 -> 55.0; +0.3 | Helped |
| Add CQMix-augmented images to task-network training (ablation) | SR4IR · Stanford Cars x8 | Top-1 accuracy: 70.3 -> 71.4; +1.1% | Helped |
Surprising findings
- Joint training of the SR and task networks with the on-training perceptual loss fails completely (3.5 mIoU), because the task network collapses to a constant output.
- Separate sequential training (S→T) achieves the best PSNR but much lower task performance than SR4IR, suggesting task-relevant high-frequency content matters more than pixel fidelity.
Limitations
- Experiments are limited to small backbones (MobileNetV3-based DeepLabV3/Faster-RCNN, ResNet18) and PASCAL VOC and two fine-grained classification datasets.
- TDSR and SOD-MTGAN were re-implemented by the authors because no public training code was available, so baseline comparisons depend on re-implementation fidelity.
Experimental setup
Datasets: PASCAL VOC2012 Stanford Cars CUB-200-2011 DIV2K
Benchmarks: PASCAL VOC2012 Stanford Cars CUB-200-2011
Metrics: mIoU mAP (COCO) Top-1 accuracy LPIPS PSNR
Baselines: Bilinear up S→T (SR then task) T→S S+T joint training TDSR SOD-MTGAN Oracle (IHR)
Builds on: perceptual loss EDSR SwinIR CutMix TDSR
Closest papers by meaning
- SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (CVPR 2024 · similarity 0.87)
- Auto-Encoded Supervision for Perceptual Image Super-Resolution (CVPR 2025 · similarity 0.87)
- Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learning (CVPR 2024 · similarity 0.86)
- Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach (CVPR 2025 · similarity 0.86)
- Unsupervised Degradation Representation Aware Transform for Real-World Blind Image Super-Resolution (AAAI 2025 · similarity 0.86)
- Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.