Research map / Image generation / Image restoration
Learning from History: Task-agnostic Model Contrastive Learning for Image Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MCLIR, a task-agnostic model contrastive paradigm where negatives come from EMA "latency" snapshots of the target model itself (Self-Prior guided Negative loss), used to retrain existing image restoration models.
Problem
Existing contrastive learning for image restoration relies on manually predefined, task-oriented negative samples that generalize poorly across tasks and architectures.
Approach
Generate negatives from the target model's own past states (an EMA-updated latency model), penalized via a VGG-feature L1 negative loss subtracted from the reconstruction loss, so no task-specific negative construction is needed.
Key results
- Retrained FFANet on SOTS-indoor reaches 39.80 dB vs 36.39 dB original (+3.41 dB); retrained DehazeFormer-B gains 0.57 dB on indoor.
- Retrained IDT averages 38.03 dB vs 37.77 dB across five deraining datasets, with +0.7 dB on SPA (48.04 vs 47.34).
- Retrained lightweight SwinIR at x4 improves average PSNR from 28.46 to 28.55 dB; Manga109 gain 0.12 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Retrain EDSR-light with SPN | EDSR-light · Manga109, ×4 super-resolution | PSNR: 30.35 -> 30.51 | Helped |
| Retrain SwinIR-light with SPN | SwinIR-light · Manga109, ×4 super-resolution | PSNR: 30.92 -> 31.04 | Helped |
| Retrain SwinIR with SPN | SwinIR · Manga109, ×4 super-resolution | PSNR: 31.67 -> 31.75 | Helped |
| Retrain IDT with SPN | IDT · SPA-Data deraining | PSNR: 47.34 -> 48.04 | Helped |
| Retrain FFANet with SPN | FFANet · SOTS-indoor dehazing | PSNR: 36.39 -> 39.80 | Helped |
| Retrain DehazeFormer-B with SPN | DehazeFormer-B · SOTS-indoor dehazing | PSNR: 37.84 -> 38.41 | Helped |
| Retrain NAFNet with SPN | NAFNet · GoPro deblurring | PSNR: 32.87 -> 32.93 | Helped |
| Use a randomly initialized, fixed negative model instead of the adaptive negative model (ablation) | SwinIR-light · ×4 super-resolution, average PSNR | PSNR: 28.55 -> 28.44 | Hurt |
| Use a pre-trained, fixed negative model instead of the adaptive negative model (ablation) | SwinIR-light · ×4 super-resolution, average PSNR | PSNR: 28.55 -> 28.51 | Hurt |
| Use multiple negative models with the four update steps together (ablation) | FFANet · SOTS-indoor dehazing | PSNR: 39.80 | Helped |
| Set balancing coefficient λ to 1e-2 or 1e-3 (ablation) | EDSR-light · ×4 super-resolution | PSNR | Hurt |
| Set balancing coefficient λ to 1e-4 (ablation) | EDSR-light · ×4 super-resolution | PSNR: 28.58 -> 28.63 | Helped |
Surprising findings
- A randomly initialized fixed negative model hurts the target SR model, while a pre-trained fixed negative model gives smaller gains than the adaptive EMA negative model.
- Retrained FFANet outperforms retrained DehazeFormer on indoor dehazing despite DehazeFormer being the more modern architecture.
Limitations
- Not evaluated on JPEG artifact removal, image denoising, or real-world scenarios.
- Large lambda values (1e-2, 1e-3) cause training collapse; hyperparameters not extensively tuned across all tasks.
- Gains are smaller for larger models (e.g., 0.05 dB for large SwinIR).
Experimental setup
Datasets: DIV2K ITS (RESIDE indoor) RESIDE-6K GoPro Rain200L Rain200H DID-Data DDN-Data SPA-Data
Benchmarks: Set14 B100 Urban100 Manga109 SOTS-indoor SOTS-mix GoPro Rain200L Rain200H DID DDN SPA
Metrics: PSNR SSIM
Baselines: EDSR SwinIR FFANet DehazeFormer IDT NAFNet CR CCR PCL MPRNet Restormer DRSformer
Builds on: contrastive regularization (CR) curricular contrastive regularization (CCR) perceptual contrastive loss (PCL) EMA
Closest papers by meaning
- Debiased All-in-one Image Restoration with Task Uncertainty Regularization (AAAI 2025 · similarity 0.88)
- IRPO: Boosting Image Restoration via Post-training GRPO (NeurIPS 2026 · similarity 0.88)
- Controlling Vision-Language Models for Multi-Task Image Restoration (ICLR 2024 · similarity 0.87)
- Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration (ICLR 2025 · similarity 0.87)
- UP-Restorer: When Unrolling Meets Prompts for Unified Image Restoration (AAAI 2025 · similarity 0.87)
- Boosting Image Restoration via Priors from Pre-trained Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.