Research map / Video / Video generation
HorizonComposer: Spatiotemporally Consistent Driving Video Editing with Enriched Traffic Semantics
Mauricio Soroco, Yuqiu Liu, Zaid Tasneem, Francesco Pittaluga, Abhishek Aich, Wuyang Chen, Manmohan Chandraker, Ziyu Jiang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HorizonComposer, an instruction-guided driving video editing method that preserves traffic semantics and spatiotemporal consistency when simulating diverse driving conditions.
Problem
General-purpose instruction-guided video editors corrupt traffic semantics when applied to global driving-condition edits, which undermines their use for stress-testing autonomous driving systems within the Operational Design Domain (ODD).
Approach
Tailors instruction-guided video editing to driving scenes by enriching traffic semantics so that global condition edits (e.g., weather, lighting) keep traffic elements consistent across frames. Only the abstract was available, so finer method details are not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dense pixel-level reward conditioning (ablation) | HorizonComposer · OOD (nuPlan) | Lane IoU: 0.5984 -> 0.6266 | Improved |
| add dense pixel-level reward conditioning (ablation) | HorizonComposer · OOD (nuPlan) | Car appearance: 0.8308 -> 0.8595 | Improved |
| add dense pixel-level reward conditioning (ablation) | HorizonComposer · In-domain | Lane IoU: 0.4276 -> 0.4904 | Improved |
| add thinking frames (temporal augmentation) (ablation) | HorizonComposer · OOD (nuPlan) | FF-alpha: 3.3927 -> 3.2185 | Improved |
| add thinking frames (ablation) | HorizonComposer · OOD (nuPlan) | Subject consistency: 0.9011 -> 0.9126 | Improved |
| add thinking frames (ablation) | HorizonComposer · OOD (nuPlan) | instruction success rate: 0.9048 -> 0.8762 | Hurt |
| add depth conditioning to RGB-only fine-tune (ablation) | HorizonComposer · In-domain | FF-alpha: 3.6594 -> 3.4901 | Improved |
| add dense rewards on top of depth conditioning (ablation) | HorizonComposer · In-domain | instruction success rate: 0.9489 -> 0.9333 | Hurt |
| plain RGB-only fine-tuning versus synthetic pipeline (ablation) | HorizonComposer · OOD (nuPlan) | Lane IoU: 0.5774 -> 0.6006 | No clear effect |
| HorizonComposer versus SyntheticPip on lane segmentation downstream | PIDNet training data · SeRM | mIoU (%): 38.45 -> 40.27 | Improved |
| add generated videos to real BDD100K training data | detector fine-tuning · BDD100K | mAP: 0.167 -> 0.175 | Improved |
| HorizonComposer versus SyntheticPip on instruction alignment and image quality | HorizonComposer · In-domain | relative change vs SyntheticPip: Succ. -1.34, Img. -4.57 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- CC-Edit — CCEdit: Creative and Controllable Video Editing via Diffusion Models (CVPR 2024) · better on 4 of 5 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Baselines: general-purpose instruction-guided video editors
Closest papers by meaning
- SceneCrafter: Controllable Multi-View Driving Scene Editing (CVPR 2025 · similarity 0.89)
- InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generation (ICCV 2025 · similarity 0.89)
- GeoV2V: Geometry-Grounded Video Diffusion Model for Driving Scene Generation (ECCV 2026 · similarity 0.88)
- HorizonForge: Driving Scene Editing with Any Trajectories and Any Vehicles (CVPR 2026 · similarity 0.88)
- DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation (CVPR 2025 · similarity 0.87)
- DriveEditor: A Unified 3D Information-Guided Framework for Controllable Object Editing in Driving Scenes (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
