Research map / Image generation / Autoregressive and latent generation
Memory-Efficient Fine-Tuning Diffusion Transformers via Dynamic Patch Sampling and Block Skipping
Sunghyun Park, Jeongho Kim, Hyoungwoo Park, Debasmit Das, Sungrack Yun, Munawar Hayat, Jaegul Choo, Fatih Porikli, Seokeon Choi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace fixed 256x256 resize with dynamic timestep-based patch sampling (ablation) | LoRA on FLUX · DreamBooth dataset | DINO: 0.7164 -> 0.7253 | Improved |
| dynamic patch sampling at 256x256 vs LoRA 512x512 (ablation) | LoRA on FLUX · DreamBooth dataset | DINO: 0.7324 -> 0.7253 | Hurt |
| add precomputed residual features when skipping blocks (30%) (ablation) | DiT-BlockSkip on FLUX · DreamBooth dataset | DINO: 0.4313 -> 0.7282 | Improved |
| add precomputed residual features when skipping blocks (50%) (ablation) | DiT-BlockSkip on FLUX · DreamBooth dataset | DINO: 0.4301 -> 0.7150 | Improved |
| skip selected blocks (cross-attention masking selection) instead of first 50% (ablation) | DiT-BlockSkip on FLUX · DreamBooth dataset | DINO: 0.6651 -> 0.7150 | Improved |
| skip selected blocks instead of last 50% (ablation) | DiT-BlockSkip on FLUX · DreamBooth dataset | DINO: 0.4808 -> 0.7150 | Improved |
| increase skip ratio from 30% to 50% with residual features (ablation) | DiT-BlockSkip on FLUX · DreamBooth dataset | DINO: 0.7282 -> 0.7150 | Hurt |
| use DiT-BlockSkip instead of HollowedNet block selection | DiT-BlockSkip 30% · DreamBooth dataset (FLUX) | DINO: 0.4899 -> 0.7194 | Improved |
| use DiT-BlockSkip instead of LISA on SANA | DiT-BlockSkip 30% · DreamBooth dataset (SANA) | DINO: 0.5226 -> 0.7388 | Improved |
| DiT-BlockSkip vs LoRA in user study (subject fidelity) | DiT-BlockSkip · User study | preference (%): 40.0% LoRA vs 37.7% ours | No clear effect |
| DiT-BlockSkip vs LoRA in user study (text fidelity) | DiT-BlockSkip · User study | preference (%): 29.4% -> 45.6% | Improved |
Closest papers by meaning
- Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers (CVPR 2025 · similarity 0.89)
- FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modification (NeurIPS 2024 · similarity 0.89)
- EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing (ICLR 2025 · similarity 0.88)
- BlockDance: Reuse Structurally Similar Spatio-Temporal Features to Accelerate Diffusion Transformers (CVPR 2025 · similarity 0.88)
- SparseDiT: Token Sparsification for Efficient Diffusion Transformer (NeurIPS 2025 · similarity 0.88)
- Dynamic Diffusion Transformer (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
