Research map / Image generation / Autoregressive and latent generation

RORem: Training a Robust Object Remover with Human-in-the-Loop

CVPR 2025 · Image generation › Autoregressive and latent generation · approach: Compositional scene generation · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds a 200K-pair object removal dataset via human-in-the-loop semi-supervised annotation with a LoRA discriminator, fine-tunes SDXL-inpainting into RORem, and distills it to 4 steps.

Problem

Existing diffusion-based object removal methods are unreliable (incomplete removal, wrong content, blurry regions) because self-supervised random-mask training is ambiguous and high-quality paired removal data is scarce.

Approach

Use human feedback on initial model outputs to train a LoRA-based discriminator that automatically labels new removal pairs over several rounds, building a large triplet dataset for supervised fine-tuning of SDXL-inpainting, then distill to 4 steps with LCM-style LoRA.

Key results

• RORem reaches a 76.2% human-judged success rate at 512x512 versus 55.8% for the best prior method (PPT), and 70.2% at 1024x1024 versus 52.4% (DesignEdit).
• The distilled 4-step RORem-4S runs in 0.50 s per image at 512 resolution (vs 4.03 s for RORem) with only a 1.4-point success-rate drop.
• Data expansion over three human/automated rounds raised the success rate of the initial model from 38.6% to 75.4% before final fine-tuning.

Measured effects reported in the paper

Surprising findings

• Self-supervised random-mask training is argued to cause ambiguity between synthesizing the masked object and restoring background, which the authors identify as the main cause of failures.
• Masking the removal region before concatenation, rather than concatenating the full source image as in ObjectDrop, reduces transparent object residuals.

Limitations

• Image quality of unmasked regions degrades due to VAE compression.
• Struggles when background contains human fingers and small faces.

Experimental setup

Datasets: RORD MULAN OpenImages DIV2K Flickr2K

Benchmarks: OpenImages-based 512x512 test set (500 pairs) OpenImages-based 1024x1024 test set (500 pairs)

Metrics: success rate (human user study) success rate (discriminator) PSNR LPIPS DINO CLIP score inference time

Baselines: LaMa SDXL-inpainting Inst-inpainting PowerPaint InstructPix2Pix CLIPAway DesignEdit

Builds on: SDXL-inpainting Latent Consistency Models ObjectDrop RORD MULAN

Closest papers by meaning

• SmartEraser: Remove Anything from Images using Masked-Region Guidance (CVPR 2025 · similarity 0.87)
• MiniMax-Remover: Taming Bad Noise Helps Video Object Removal (NeurIPS 2025 · similarity 0.86)
• GeoRemover: Removing Objects and Their Causal Visual Artifacts (NeurIPS 2025 · similarity 0.83)
• Attentive Eraser: Unleashing Diffusion Model’s Object Removal Potential via Self-Attention Redirection Guidance (AAAI 2025 · similarity 0.83)
• IRPO: Boosting Image Restoration via Post-training GRPO (NeurIPS 2026 · similarity 0.83)
• CLIPAway: Harmonizing focused embeddings for removing objects via diffusion models (NeurIPS 2024 · similarity 0.82)