Research map / Image generation / Autoregressive and latent generation

SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: High-resolution diffusion synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SEGA, a training-free method that adapts RoPE attention scaling per dimension using the spectral energy of the current latent at each denoising step, improving high-resolution extrapolation of Flux and Qwen DiTs.

Problem

Diffusion transformers degrade when generating images beyond their training resolution, because RoPE extrapolation dilutes attention; existing uniform attention scaling trades off global structure against fine detail.

Approach

Derives per-RoPE-dimension attention scaling from the spectral energy of the current intermediate latent (axis-wise and radial power spectra), with a global amplitude set by spectral flatness, so scaling is content-aware and evolves over denoising steps.

Key results

• On Flux at 4096x4096, SEGA reaches FID 150.05 and PickScore 23.18 versus 183.33 and 20.31 for the base model and 151.74 for I-Max (best baseline FID).
• On Qwen at 4096x2048, SEGA improves FID to 147.51 versus 159.38 for DyPE (second best).
• SEGA outperforms all baselines across metrics at 4096^2 on Qwen, and reaches FID 221.99 at 5120^2 on Flux versus 225.30 for DyPE.

Surprising findings

• YaRN frequently causes spatial structure collapse and layout confusion in 2D image generation, so the authors use NTK as the default base method instead.
• Multi-stage guidance methods (HiFlow, I-Max, ScaleDiff) do not uniformly beat direct-inference baselines; several have lower MSQ or higher FID_p than SEGA's direct-inference counterparts.

Limitations

• SEGA does not extend RoPE's positional range and relies on an underlying extrapolation method (NTK by default), inheriting its priors.
• Perceptual quality still degrades at extreme extrapolation factors (up to 8192^2), limited by model capacity.
• Ultra-high-resolution generation remains computationally expensive due to quadratic attention cost.

Experimental setup

Datasets: Aesthetic-4K

Benchmarks: Aesthetic-4K Zero-Shot benchmark

Metrics: ImageReward PickScore CLIP Score MUSIQ FID FID_p HPSv2 CLIP-IQA

Baselines: NTK YaRN DyPE UltraImage PI HiFlow I-Max ScaleDiff DemoFusion FreCaS FreeScale DiffuseHigh

Builds on: RoPE YaRN NTK-aware scaling Position Interpolation DyPE

Closest papers by meaning

• ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion (NeurIPS 2025 · similarity 0.90)
• FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.89)
• LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding (NeurIPS 2025 · similarity 0.89)
• CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up (NeurIPS 2025 · similarity 0.89)
• DiTFastAttn: Attention Compression for Diffusion Transformer Models (NeurIPS 2024 · similarity 0.88)
• ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models (NeurIPS 2026 · similarity 0.88)