Research map / Image generation / Text-to-image diffusion
Token-to-Token Alignment of Text Embeddings for Semantic Blending
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Token-to-Token alignment, which uses LLM-based structural rephrasing plus soft token-level embedding alignment so that linear interpolation of text embeddings yields smooth semantic transitions for image blending and continuous editing with diffusion models.
Problem
Linear interpolation between text prompt embeddings produces inconsistent, entangled intermediate images because corresponding concepts are not consistently tokenized or positioned across prompts, hindering continuous semantic blending and editing.
Approach
Two-stage alignment: an LLM rewrites prompt pairs into a shared JSON schema with parallel phrasing (structural alignment), then a cosine-similarity soft alignment matrix with field masking and a positional Gaussian bias projects source token embeddings onto target token layout (embedding alignment); linear interpolation of aligned embeddings then conditions a frozen diffusion model without any tra
Key results
- On BlendBench, T2T with Flux2-klein achieves MUSIQ 73.27 and FID 85.06, versus 146.80 FID for DiffMorpher and 129.53 for FreeMorph.
- On PIE-Bench, T2T FIBO-edit reaches Norm CLIP-Dir 0.393 and continuity 1.46, the best among editing baselines (Kontinuous Kontext 0.259 and 1.33).
- Kontinuous Kontext and morphing baselines retain better smoothness: Kontinuous Kontext has the lowest delta_smooth (0.246), and DiffMorpher has the lowest blending PPL on both benchmarks, while T2T is weaker on PPL.
Surprising findings
- Morphing-based baselines achieve better trajectory smoothness (PPL) than the proposed method, which the authors attribute to interpolation in perceptual space.
Limitations
- Quality depends on the LLM structural alignment; errors or ambiguities propagate to trajectories.
- Linear interpolation may not capture highly non-linear semantic transformations.
- Smoothness (PPL) lags morphing-based baselines on blending benchmarks.
Experimental setup
Datasets: Morph4Data PIE-Bench
Benchmarks: Morph4Data BlendBench PIE-Bench
Metrics: PPL MUSIQ FID delta_smooth continuity Norm CLIP-Dir
Baselines: DiffMorpher FreeMorph VibeSpace Kontinuous Kontext SliderEdit GRAG
Builds on: FLUX.2 FIBO FreeMorph Kontinuous Kontext
Closest papers by meaning
- Text Embedding is Not All You Need: Attention Control for Text-to-Image Semantic Alignment with Text Self-Attention Maps (CVPR 2025 · similarity 0.90)
- Aligning Text to Image in Diffusion Models is Easier Than You Think (NeurIPS 2025 · similarity 0.89)
- Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis (NeurIPS 2024 · similarity 0.88)
- Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactions (ICML 2026 · similarity 0.87)
- CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching (NeurIPS 2024 · similarity 0.87)
- PromptCoT: Align Prompt Distribution via Adapted Chain-of-Thought (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.