Research map / Image generation / Text-to-image diffusion
Head-wise Adaptive Rotary Positional Encoding for Fine-Grained Image Generation
Jiaye Li, Baoyou Chen, Hui Li, Zilong Dong, Jingdong Wang, Siyu Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace RoPE with HARoPE | DiT-B/2 · ImageNet 256 | FID-50K: 9.49 -> 8.90 | Improved |
| make the adaptation matrix head-specific (ablation) | RoPE + SVD (DiT-B/2) · ImageNet 256 | FID-50K: 8.93 -> 8.90 | Improved |
| make normal matrix head-specific (ablation) | RoPE + normal-matrix · ImageNet 256 | FID-50K: 9.28 -> 9.03 | Improved |
| use SVD parameterization instead of normal or orthogonal matrix (ablation) | HARoPE · ImageNet 256 | FID-50K: 9.28 / 9.31 -> 8.93 | Improved |
| add head-wise SVD rather than parameter-matched shared RoPE+SVD (ablation) | FLUX.1-dev · GenEval Overall 2048 | GenEval overall: 0.731 -> 0.756 | Improved |
| add shared RoPE+SVD (no head-wise) (ablation) | FLUX.1-dev · GenEval Overall 2048 | GenEval overall: 0.735 -> 0.731 | No clear effect |
| tune RoPE theta_base per frequency instead of HARoPE (ablation) | DiT-B/2 100k steps · ImageNet 256 | FID: 20.77 (10,000) vs 20.45 (5,000) vs 19.01 HARoPE | No clear effect |
| replace RoPE with HARoPE at 512x512 | FLUX.1-dev · GenEval Overall 512 | GenEval overall: 0.765 -> 0.758 | Hurt |
| replace RoPE with HARoPE at 1024x1024 | FLUX.1-dev · GenEval Overall 1024 | GenEval overall: 0.757 -> 0.771 | Improved |
| replace RoPE with HARoPE at 2048x2048 | FLUX.1-dev · GenEval Position | Position score: 0.370 -> 0.455 | Improved |
| replace RoPE with HARoPE | FLUX.1-dev · T2I-CompBench 2D-Spatial 1024 | score: 0.307 -> 0.337 | Improved |
| replace RoPE with HARoPE | SD3-medium · MS-COCO 256 | FID: 5.35 -> 5.22 | Improved |
Closest papers by meaning
- ComRoPE: Scalable and Robust Rotary Position Embedding Parameterized by Trainable Commuting Angle Matrices (CVPR 2025 · similarity 0.88)
- Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Vision-Language Models (ICML 2026 · similarity 0.87)
- Revisiting Multimodal Positional Encoding in Vision–Language Models (ICLR 2026 · similarity 0.86)
- DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers (NeurIPS 2026 · similarity 0.85)
- UNIC-Adapter: Unified Image-instruction Adapter with Multi-modal Transformer for Image Generation (CVPR 2025 · similarity 0.85)
- A Circular Argument: Does RoPE need to be Equivariant for Vision? (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
