Research map / Image generation / Text-to-image diffusion
Learning Subject-Aware Cropping by Outpainting Professional Photos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GenCrop, a weakly-supervised subject-aware cropper trained on pseudo-labeled crops generated by outpainting Unsplash stock photos with Stable Diffusion inpainting.
Problem
Subject-aware image cropping normally requires large manually annotated crop datasets; the paper asks whether professional stock photos can supply crop supervision without new annotations, despite the uncropped originals being unavailable.
Approach
Treat each professional stock photo as a pseudo-label crop and synthesize the missing uncropped context by outpainting it with a text-conditioned diffusion inpainting model, yielding cropped-uncropped training pairs; a CACNet-inspired regressor with subject-boundary loss is trained on these pairs.
Key results
- On FCDB+FLMS GenCrop reaches IoU 0.7334 and Disp 0.0687, versus 0.5783/0.1114 for VFN trained on Unsplash, and within 0.014 IoU of supervised HCIC.
- On SACD GenCrop achieves IoU 0.7301 and Disp 0.0632, beating HCIC (0.7120) and GAIC (0.7124) trained on GAICD.
- Across six subject classes GenCrop-6 has the fewest violations on five of six subjects and overall (mean 4.5% of images vs 51.2% for VFN and 8.8% for HCIC).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GenCrop's outpainting-generated training data instead of VFN's weak-supervision approach | Weakly supervised image cropping · FCDB+FLMS | IoU: 0.5783 -> 0.7334 | Helped |
| Use GenCrop's outpainting-generated training data instead of VFN's weak-supervision approach | Weakly supervised image cropping · SACD | IoU: 0.6555 -> 0.7301 | Helped |
| Add the subject boundary loss (ablation) | GenCrop · FCDB+FLMS | IoU: 0.7145 -> 0.7334 | Helped |
| Use subject mask and bounding-box information, with its dependent boundary loss (ablation) | GenCrop · SACD | IoU: 0.7105 -> 0.7301 | Helped |
| Filter training data to the target subject category rather than using generic images (ablation) | GenCrop · FCDB+FLMS | IoU: 0.6861 -> 0.7334 | Helped |
| Remove poorly outpainted images with both quality filters (ablation) | GenCrop · FCDB+FLMS | IoU: 0.6987 -> 0.7334 | Helped |
| Reduce the stock-image source set to 10000 images (ablation) | GenCrop · FCDB+FLMS | IoU: 0.7334 -> 0.7174 | Hurt |
| Reduce the stock-image source set to 1000 images (ablation) | GenCrop · FCDB+FLMS | IoU: 0.7334 -> 0.7007 | Hurt |
| Reduce the stock-image source set to 100 images (ablation) | GenCrop · FCDB+FLMS | IoU: 0.7334 -> 0.6865 | Hurt |
| Add depth input to the crop model (ablation) | GenCrop · FCDB+FLMS | IoU: 0.7334 -> 0.7226 | Hurt |
| Add edge input to the crop model (ablation) | GenCrop · SACD | IoU: 0.7301 -> 0.7225 | Hurt |
| Replace ResNet-50 with ResNet-18 (ablation) | GenCrop · FCDB+FLMS | IoU: 0.7334 -> 0.7181 | Hurt |
Surprising findings
- Training on outpainted images from only 100 stock photos still outperforms VFN trained on 52K images.
- Training on humans only (GenCrop-H) transfers reasonably to cats, dogs, birds, horses and cars, and GenCrop-6 generalizes across categories.
Limitations
- Sparse one-crop-per-image pseudo-labels do not reproduce dense crop-ranking annotations; ranking performance is weak on GAICD.
- Stable Diffusion outpainting artifacts such as extra subjects, composites and frames require heuristic and CNN filtering; roughly 20% of outpainted portraits discarded.
- Generic-image performance lags supervised methods trained on CPC, reflecting the distribution mismatch between Unsplash and crowd-sourced benchmarks.
Experimental setup
Datasets: Unsplash FCDB FLMS SACD CPC GAICD COCO
Benchmarks: FCDB+FLMS SACD Unsplash six-subject test sets GAICD FCDB generic
Metrics: IoU boundary displacement SRCC Acc5 Acc10 violation rate
Baselines: VFN HCIC CACNet GAIC CGS LVRN SAC-Net FRCNN-m
Builds on: Stable Diffusion CACNet VFN RoDAlign
Closest papers by meaning
- ProCrop: Learning Aesthetic Image Cropping from Professional Compositions (AAAI 2026 · similarity 0.91)
- Cropper: Vision-Language Model for Image Cropping through In-Context Learning (CVPR 2025 · similarity 0.87)
- Spatial-Semantic Collaborative Cropping for User Generated Content (AAAI 2024 · similarity 0.86)
- Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping (CVPR 2026 · similarity 0.83)
- Training-free Detection of AI-generated images via Cropping Robustness (NeurIPS 2025 · similarity 0.83)
- Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generator (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.