Research map / Vision-language and multimodal / Multimodal LLMs
LangDriveCTRL: Natural Language Controllable Driving Scene Editing with Multi-modal Agents
Yun He, Francesco Pittaluga, Ziyu Jiang, Matthias Zwicker, Manmohan Chandraker, Zaid Tasneem
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Behavior Reviewer feedback loop (ablation) | LangDriveCTRL behavior editing · Multi-object behavior instructions (70) | overall success (%): 34.29 -> 51.43 | Improved |
| add counterfactual behavior generation (ablation) | Behavior Editing Agent · Multi-object behavior instructions (70) | overall success (%): 47.14 -> 51.43 | Improved |
| add counterfactual behavior generation to reject infeasible instructions (ablation) | Behavior Editing Agent · Unreasonable instructions (30) | rejection rate (%): 0.00 -> 93.33 | Improved |
| apply video diffusion tool once to coarse render (ablation) | LangDriveCTRL rendering · Insertion/replacement (video refinement ablation) | FID: 43.54 -> 36.83 | Improved |
| apply video diffusion tool once to coarse render (ablation) | LangDriveCTRL rendering · Insertion/replacement (video refinement ablation) | appearance alignment (%): 87.69 -> 65.47 | Hurt |
| add Video Reviewer Agent on top of diffusion tool (ablation) | LangDriveCTRL rendering · Insertion/replacement (video refinement ablation) | appearance alignment (%): 65.47 -> 85.28 | Improved |
| add Video Reviewer Agent on top of diffusion tool (ablation) | LangDriveCTRL rendering · Insertion/replacement (video refinement ablation) | FID: 36.83 -> 36.78 | No clear effect |
| video diffusion refinement of coarse render (ablation) | LangDriveCTRL · Waymo hallucination analysis | 3D consistency: 74.07 -> 72.64 | Hurt |
| use coarse scene-graph render without diffusion vs ChatSim (ablation) | LangDriveCTRL (Coarse) · Waymo editing (30 scenes) | behavior alignment (%): 26.64 -> 73.18 | Improved |
| add edited frames to real training data | BEVFormer · Waymo BEVFormer detection | AP@0.5: 0.0533 -> 0.0635 | Improved |
| use VLM-based Object Grounding Agent instead of Grounding SAM | object query · Object grounding (50 queries) | accuracy (%): 44.00 -> 84.00 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- 4DLangSplat — 4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models (CVPR 2025) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment (CVPR 2025 · similarity 0.84)
- SceneCrafter: Controllable Multi-View Driving Scene Editing (CVPR 2025 · similarity 0.83)
- DriveEditor: A Unified 3D Information-Guided Framework for Controllable Object Editing in Driving Scenes (AAAI 2025 · similarity 0.83)
- Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents (CVPR 2024 · similarity 0.83)
- TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024 · similarity 0.82)
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
