Research map / Image generation / Editing and style transfer

Why Do DiT Editors Drift? Plug-and-Play Low Frequency Alignment in VAE Latent Space

NeurIPS 2026 · Image generation › Editing and style transfer · approach: Inversion-based diffusion editing · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VAE-LFA, a training-free plug-and-play module that aligns low-frequency VAE latent statistics to an EMA of previous editing rounds to curb semantic drift in multi-turn DiT image editing, for both white-box and black-box editors.

Problem

Multi-turn instruction-based image editing with DiT editors accumulates progressive semantic drift and quality degradation, and existing remedies need retraining or internal access, excluding black-box API editors.

Approach

Decomposes per-round latent drift into VAE and DiT components via frequency analysis, finds DiT induces dominant low-frequency drift, and corrects only low-frequency channel-wise mean and std toward an EMA anchor while leaving high-frequency residuals untouched.

Key results

• On FLUX.2 no-op round 10, +LFA lowers LPIPS from 0.36 to 0.17 and raises SSIM from 0.81 to 0.93 on the creature/photograph subset.
• Bootstrap 95% CI for round-10 no-op LPIPS improvement across 480 paired observations is [0.1139, 0.1487].
• On Qwen Image 2.0 long-chain round 10 (photograph, clear object), +LFA raises DINO from 0.29 to 0.37 and VLM score from 55.1 to 60.9 (clear object).

Surprising findings

• Removing VAE round trips reduces high-frequency drift but leaves low-frequency drift, and VAE round trips partly regularize low-frequency drift, challenging the view that repeated VAE cycles dominate error accumulation.
• Aligning only high-frequency components helps little, and aligning both can over-constrain valid edits in cycle editing.

Limitations

• Low-frequency alignment can mildly suppress intended large global changes under strong prompts (e.g., black-to-white image).
• Evaluation relies on a small in-house 120-image set and a VLM judge.
• Black-box setting depends on an external SD-VAE that differs from the editor's native latent space.

Experimental setup

Datasets: 120-image in-house evaluation set (illustration/photograph; creature/architecture/scenery)

Benchmarks: in-house 120-image no-op, cycle, and long-chain editing sets

Metrics: LPIPS L1 SSIM DINOv3 subject consistency VLM score (0-100)

Baselines: FLUX.2 SD3-UE Qwen Image 2.0 Seedream 4.0 FreqEdit REED-VAE Layer-wise Memory

Builds on: FLUX.2 Stable Diffusion 3 SD-VAE-ft-ema EdiVal-Agent

Closest papers by meaning

• Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.89)
• Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editing (NeurIPS 2024 · similarity 0.87)
• Stable Flow: Vital Layers for Training-Free Image Editing (CVPR 2025 · similarity 0.87)
• Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps (NeurIPS 2024 · similarity 0.87)
• DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing (NeurIPS 2025 · similarity 0.87)
• Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation (CVPR 2024 · similarity 0.86)