Research map / Video / Video generation
Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LA-LQR, a reduced-order linear-quadratic regulator that steers activations of text-to-video diffusion transformers within a low-dimensional contrastive subspace to reduce unsafe generations while preserving quality.
Problem
Activation steering of text-to-video models is typically coarse and non-anticipative, causing oversteering and content degradation; full-space optimal control is infeasible because video DiT activations have tens of millions of dimensions.
Approach
Project DiT activations onto a low-rank subspace from contrastive prompt pairs, estimate local linear latent dynamics with Jacobian-vector/vector-Jacobian products, and solve a latent LQR problem to get closed-loop, timestep- and layer-specific minimum-norm steering toward feature setpoints.
Key results
- On T2VSafetyBench with Wan2.1-14B, LA-LQR reduces average violation rate to 14.6% vs 36.9% for the unsteered model and 26.1% for the best baseline.
- On SafeSora with HunyuanVideo-1.5, average violation rate drops to 12.2% from 31.5% unsteered, with VBench subject consistency 0.967 vs 0.949.
- LA-LQR has lower CAPS (0.61 on Wan, 0.65 on Hunyuan) than baselines, which the authors attribute to larger semantic changes needed for safety.
Surprising findings
- Video-only steering induces the target concept but is more prone to oversteering and artifacts than text-only steering, according to the appendix.
Limitations
- Memory and storage overhead for projection bases and projected dynamics across layers and timesteps.
- Performance depends on latent dimension and contrastive-subspace quality; missed concepts can be discarded by projection.
- Assumes white-box access and local linear dynamics, which may degrade for adversarial prompts.
Experimental setup
Datasets: T2VSafetyBench SafeSora
Benchmarks: T2VSafetyBench SafeSora VBench
Metrics: violation rate VBench subject consistency CAPS background consistency motion smoothness dynamic degree
Baselines: Slider SAFREE Unlearning Wan2.1-T2V-14B (unsteered) HunyuanVideo-1.5 (unsteered)
Builds on: linear quadratic regulator activation steering randomized SVD T2VSafetyBench
Closest papers by meaning
- T2V-Turbo-v2: Enhancing Video Model Post-Training through Data, Reward, and Conditional Guidance Design (ICLR 2025 · similarity 0.87)
- T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback (NeurIPS 2024 · similarity 0.86)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.85)
- Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior (NeurIPS 2026 · similarity 0.85)
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 · similarity 0.85)
- VideoDirector: Precise Video Editing via Text-to-Video Models (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.