Research map / Detection and segmentation / Open-set and few-shot detection
Learning Degradation-Independent Representations for Camera ISP Pipelines
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiRNet, a self-supervised VAE-style encoder trained with multi-view mutual information maximization on ISP-degraded images, refined by a guided alignment network toward degradation-free representations for restoration, detection and segmentation.
Problem
Camera ISP pipelines introduce compounded, partly unknown degradations (sensor noise, demosaicing, compression, ISP hyperparameter errors) that hurt generalization of image restoration networks and downstream vision tasks, and paired clean/degraded data is hard to obtain.
Approach
Learn a degradation-independent representation (DiR) self-supervised by maximizing multi-view mutual information between two differently degraded views of the same image, then refine it with a guided alignment network that aligns it to a degradation-free representation reference learned from clean images, jointly trained with a downstream task network.
Key results
- On real-world ISP-degraded images, the DiR method achieves the best NIQE/NRQM/BIQA on all four datasets in Table 1 (e.g., PolyU-Real NIQE 6.57, NRQM 4.91, BIQA 51.9).
- On COCO object detection, mAP@0.5 reaches 63.9 vs 62.8 for sequential-tuned ISP and 61.0 for attention-aware ISP.
- On COCO instance segmentation, mAP@0.5 is 56.3 vs 54.2 for sequential-tuned ISP; on ADE20k with SegFormer, mIoU is 42.4 vs 37.8 for DNF.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn DiR representations for blind restoration | ISP-degraded image restoration · PolyU-Real, NC12, Nam, and DND | NIQE, NRQM, and BIQA | Mixed |
| use DiR learning instead of Sequential-tuned | MS COCO object detection · MS COCO | mAP@0.5: 62.8 -> 63.9 | Helped |
| use DiR learning instead of Sequential-tuned | MS COCO image segmentation · MS COCO | mAP@0.5: 54.2 -> 56.3 | Helped |
| add the alignment network to baseline DiR (ablation) | DiR learning · Set14 | PSNR: 22.4 -> 23.9 | Helped |
| add the alignment network to baseline DiR (ablation) | DiR learning · BSD100 | PSNR: 23.8 -> 24.6 | Helped |
| add the pilot DfR to alignment (ablation) | DiR learning · Set14 | PSNR: 23.9 -> 24.1 | Helped |
| add the pilot DfR to alignment (ablation) | DiR learning · BSD100 | PSNR: 24.6 -> 25.0 | Helped |
Limitations
- Degradation-free reference relies on high-quality real camera images that are not strictly degradation-free.
- Restoration evaluation uses only non-reference metrics since no ground truth exists for real images.
- Training relies on synthetic ISP degradations, whose match to real unknown degradations is not fully verified.
Experimental setup
Datasets: DIV2K MS COCO ADE20k PolyU-Real NC12 Nam DND Set14 BSD100
Benchmarks: PolyU-Real NC12 Nam DND MS COCO ADE20k Set14 BSD100
Metrics: NIQE NRQM BIQA mAP mIoU mAcc aAcc PSNR LPIPS
Baselines: BM3D CBDNet LIR MPRNet Uformer SCPGab DIL SDAP Attention-aware Sequential-tuned ISP PyNET DNF
Builds on: InfoVAE VAE multi-view mutual information maximization Jensen-Shannon mutual information estimator
Closest papers by meaning
- DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor (NeurIPS 2024 · similarity 0.86)
- Universal Image Restoration Pre-training via Degradation Classification (ICLR 2025 · similarity 0.84)
- Unsupervised Degradation Representation Aware Transform for Real-World Blind Image Super-Resolution (AAAI 2025 · similarity 0.84)
- DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution (CVPR 2025 · similarity 0.84)
- Degradation-Aware Feature Perturbation for All-in-One Image Restoration (CVPR 2025 · similarity 0.83)
- AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.