Research map / Image generation / Text-to-image diffusion
One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains sparse autoencoders on residual updates of four cross-attention transformer blocks in SDXL Turbo's one-step U-net, shows the features generalize to multi-step SDXL Turbo, SDXL base and FLUX Schnell, and introduces RIEBench for feature-transfer image editing.
Problem
Text-to-image diffusion models are black boxes, and sparse autoencoder interpretability methods developed for LLMs have not been applied to their intermediate representations.
Approach
Train TopK SAEs on the residual updates of selected transformer blocks in SDXL Turbo's 1-step denoising U-net, then show the learned features transfer without retraining to multi-step SDXL Turbo, SDXL base and FLUX Schnell, and can causally steer generation via feature add/subtract interventions.
Key results
- SAEs trained on 1-step SDXL Turbo keep high explained variance across 4-step SDXL Turbo and 20-step SDXL base denoising without retraining.
- On RIEBench, SAE features match neuron interventions in most edit categories while requiring several orders of magnitude fewer features; steering and SAEs significantly outperform neurons on change content, color, material and style.
- Block analysis shows down.2.1 acts as a composition block and up.0.1 as a style block, with up.0.1 most relevant for color, material, background and style edits.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train SAEs on one-step activations and apply them without additional training to four-step denoising | SDXL Turbo SAEs · SDXL Turbo 4-step generation | explained variance | No clear effect |
| Train SAEs on one-step SDXL Turbo activations and apply them to the multi-step base model | SDXL Turbo SAEs · SDXL base multi-step generation | explained variance | No clear effect |
| Increase the sparsity parameter k (ablation) | SDXL Turbo SAEs · SAE training | explained variance | Helped |
| Increase the SAE expansion factor (ablation) | SDXL Turbo SAEs · SAE training | explained variance | Helped |
| Use SAE feature transfer instead of neuron transfer | SDXL Turbo RIEBench · change content, change color, change material, and change style | CLIP similarity increase | Helped |
| Use SAE features or steering instead of neurons | SDXL Turbo RIEBench · change object | CLIP similarity increase | Mixed |
| Use feature transfer with SAE features, neurons, or activation steering | SDXL Turbo RIEBench · most edit categories | CLIP similarity increase at a given LPIPS budget | No clear effect |
| Apply SAE feature transfer for change-pose edits | SDXL Turbo RIEBench · change pose | CLIP similarity increase | No clear effect |
| Use multi-layer rather than single-layer interventions (ablation) | FLUX Schnell SAE interventions · change object, add object, change content, color, material, background, and style | CLIP similarity increase | Helped |
| Use layer 18 neuron interventions | FLUX Schnell RIEBench · all edit categories | CLIP similarity increase | No clear effect |
Surprising findings
- Despite down.2.1 having strong qualitative impact, it is rarely selected in RIEBench edits, and mid.0 has low causal impact.
- Adding features naively under classifier-free guidance makes their effects inhibit each other, requiring subtraction in the unconditional pass.
Limitations
- Method focuses on individual blocks and may miss features requiring combinations of blocks.
- Edit success varies by feature type and target image; global and compositional features remain hard.
- Change pose and delete object edits perform poorly for all methods.
Experimental setup
Datasets: LAION-COCO LAION-5B PIEBench
Benchmarks: RIEBench PIEBench
Metrics: explained variance LPIPS CLIP similarity feature overlap (cosine similarity)
Baselines: neuron intervention activation steering
Builds on: sparse autoencoders TopK sparse autoencoder SDXL Turbo PIEBench Grounded SAM2
Closest papers by meaning
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.89)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.89)
- Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.89)
- Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Scaling and evaluating sparse autoencoders (ICLR 2025 · similarity 0.88)
- Low-Rank Adapting Models for Sparse Autoencoders (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.