Research map / Video / Video generation

SteerVTE: Seamless Video Text Editing with Style and Glyph Control

NeurIPS 2026 · Video › Video generation · approach: Diffusion-based video editing · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SteerVTE, a frozen-DiT video text editing model with a Text Context Adapter (mask, VLM style encoder, dual-granularity glyph), trained via a three-stage curriculum with a GLAS loss on a 1M-triplet dataset, and evaluated on the new VTE-Bench.

Problem

Video text editing requires stroke-level precision in small text regions while keeping cross-frame accuracy, temporal coherence, and style fidelity; existing image text editors are word-level and video foundation models have weak text priors, with no dedicated benchmark or large training data.

Approach

Freeze a pretrained text-to-video DiT (VACE 14B) and inject editing signals through a lightweight Text Context Adapter: spatiotemporal mask conditioning, a native-resolution VLM style encoder, and dual-granularity (line- and character-level) glyph priors, plus a GLAS loss combining latent spatial-focal reweighting with OCR-based supervision, trained by a three-stage image-to-video curriculum.

Key results

• On VTE-Bench-Real, SteerVTE reaches 77% sentence accuracy vs. 32% for Seedance 2.0 and NED 0.958 vs. 0.611.
• On VTE-Bench-Synth, SteerVTE reaches 61% accuracy, NED 0.962 and FVD 14.55, versus FLUX-Text FVD 314.9.
• User study Quality Rate: SteerVTE 71% vs. Seedance 2.0 23% on real videos.

Surprising findings

• The original V2V paradigm achieves only 13% sentence accuracy, and adding a mask alone raises it by 11 points.
• Adding an OCR loss alone raises text accuracy by about 14 points but degrades style consistency, so the spatial-focal term is needed alongside it.

Limitations

• Supports only English and single-line editing.
• Sentence accuracy remains at 61% on synthetic and 77% on real benchmarks, so single-character errors persist.
• Evaluation on real videos has no ground truth and relies on a five-person user study with a small sample.

Experimental setup

Datasets: SynthTE-Easy SynthTE-Hard RealTE AnyWord-3M OpenSora-Plan background videos Google Fonts

Benchmarks: VTE-Bench-Synth VTE-Bench-Real

Metrics: precision recall F-score NED sentence accuracy SSIM PSNR FID FVD Style-Sim

Baselines: FLUX-Text VACE VideoPainter VIVA UniVideo Kiwi-Edit Seedance 2.0

Builds on: flow matching VACE LoRA PP-OCRv3 Qwen2.5-VL

Closest papers by meaning

• ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing (NeurIPS 2026 · similarity 0.90)
• TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024 · similarity 0.88)
• VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling (NeurIPS 2026 · similarity 0.87)
• Textualize Visual Prompt for Image Editing via Diffusion Bridge (AAAI 2025 · similarity 0.87)
• Text-Vision Co-Instructed Image Editing (NeurIPS 2026 · similarity 0.87)
• VicEdit: Learning to Edit Videos from Visual In-Context Examples (NeurIPS 2026 · similarity 0.87)