AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

ControlNet-XS: Rethinking the Control of Text-to-Image Diffusion Models as Feedback-Control Systems

Denis Zavadski, Johann-Friedrich Feiden, Carsten Rother

ECCV 2024 · Image generation › Text-to-image diffusion · approach: Text-driven synthesis and semantic guidance · 56 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace ControlNet with ControlNet-XS (high-frequency, large-bandwidth feedback, 55M params) for depth controlControlNet · COCO val depth controlFID: 19.01 -> 16.36Improved
replace ControlNet with ControlNet-XS for depth control fidelityControlNet · COCO val depth controlMSE-depth (x10^3): 29.1 -> 19.6Improved
replace ControlNet with ControlNet-XS for canny edge controlControlNet · COCO val canny edge controlFID: 21.18 -> 15.13Improved
replace ControlNet with ControlNet-XS for semantic map controlControlNet-XS vs competitors · COCO-Stuff val semantic mapFID: 22.26 -> 17.70 (vs Uni-ControlNet)Improved
use ControlNet-XS for human pose keypoint controlControlNet-XS · COCO val human poseFID: 23.82 -> 23.58 (vs ControlNet)No clear effect
add control signals in the controller encoder part (Type B vs Type A) (ablation)ControlNet-XS · COCO val depth controlFID: 17.11 -> 16.36Improved
add mirrored decoder connections (Type C vs Type B) (ablation)ControlNet-XS · COCO val depth controlFID: 16.36 -> 16.24No clear effect
reduce controller from 491M to 55M parameters (ablation)ControlNet-XS · COCO val depth controlFID: 16.91 -> 16.36Improved
reduce controller from 55M to 1.7M parameters (ablation)ControlNet-XS · COCO val depth controlMSE-depth (x10^3): 19.6 -> 29.9Hurt
reduce controller from 55M to 11.7M parameters (ablation)ControlNet-XS · COCO val depth controlFID: 16.36 -> 17.90Hurt
use a smaller controller to reduce semantic bias (ablation)Depth control with unrelated prompt · Qualitative cake-street-scene testsemantic biasImproved
replace ControlNet with ControlNet-XS (55M)Depth control inference/training · Efficiencyinference time: 1min 11sec -> 38secImproved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 4 of 4 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy