Research map / Image generation / Text-to-image diffusion
TiNO-Edit: Timestep and Noise Optimization for Robust Diffusion-Based Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Optimizes diffusion timesteps and initial noise of Stable Diffusion at inference time, using latent-domain CLIP and VGG losses, to make image editing (text, reference, stroke, composition, DreamBooth/Textual Inversion) more faithful and controllable.
Problem
Controllable, predictable diffusion-based image editing with pre-trained text-to-image models is hard; prior methods optimize weights, prompts, or features, but the noise and starting timesteps are typically fixed or hand-tuned, and pixel-domain losses are slow.
Approach
Treat the initial Gaussian noise and all per-step diffusion timesteps of SD Img2Img as optimization variables, driven by a new set of loss functions (LatentCLIP semantic loss, LatentVGG perceptual loss) computed directly in the VAE latent space, which speeds optimization versus pixel-domain losses.
Key results
- On pure text-guided editing, TiNO-Edit achieves CLIP-I 0.924 and DINO-I 0.874, the best among compared methods (e.g., SINE CLIP-I 0.912, EDICT DINO-I 0.838).
- On stroke-guided editing, TiNO-Edit reports DINO-I 0.978 and CLIP-I 0.959, above BLD and SDEdit variants.
- Optimizing both timesteps and noise is necessary: fixing timesteps causes output degradation as optimization proceeds (Fig. 10), per the authors' ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize diffusion timesteps and input noise instead of keeping them fixed (ablation) | TiNO-Edit · Image editing | editing robustness across optimization steps | Helped |
| Keep diffusion timesteps constant rather than optimizing them (ablation) | TiNO-Edit · Image editing | output quality over optimization steps | Hurt |
| Do not optimize the input noise (ablation) | TiNO-Edit · Image editing | artifact prevalence and image preservation | Hurt |
| Remove the editing-region mask (ablation) | TiNO-Edit · Image editing | preservation of global image structure | Hurt |
| Use LatentCLIP and LatentVGG instead of pixel-domain CLIP and VGG losses (ablation) | TiNO-Edit · Image editing, including custom concepts | editing quality | Helped |
| Use LatentCLIP and LatentVGG instead of pixel-domain CLIP and VGG losses (ablation) | TiNO-Edit · Textual Inversion custom concepts | editing quality | Helped |
Limitations
- Evaluation relies on a self-created 200-sample test set and automatic CLIP/DINO metrics without human evaluation.
- Requires per-image optimization (50 steps over 10 denoising steps), which adds inference cost.
- Some baselines are only shown qualitatively; quantitative comparisons are limited to a subset of methods.
Experimental setup
Datasets: LAION-5B test set of 200 samples
Benchmarks: 200-sample in-house editing test set
Metrics: CLIP-T CLIP-I CLIP-I* DINO-I
Baselines: Text2LIVE DisentanglementDiffusion Pix2Pix-Zero SINE EDICT InstructPix2Pix Plug-and-Play Null-text Inversion Visual Concept Translator GLIGEN Paint-by-Example Blended Latent Diffusion
Builds on: DDIM Stable Diffusion SDEdit CLIP Textual Inversion
Closest papers by meaning
- Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editing (NeurIPS 2024 · similarity 0.89)
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.89)
- An Edit Friendly DDPM Noise Space: Inversion and Manipulations (CVPR 2024 · similarity 0.89)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.89)
- Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks (AAAI 2024 · similarity 0.88)
- VersaFusion: A Versatile Diffusion-Based Framework for Fine-Grained Image Editing and Enhancement (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.