Research map / Large language models / Agents, code and math

DeEscalWild: A Real-World Benchmark for Automated De-Escalation Training with SLMs

NeurIPS 2026 · Large language models › Agents, code and math · approach: Agent instruction-following benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DeEscalWild, a benchmark of 1,500 anonymized transcripts of in-the-wild police-civilian interactions, and shows QLoRA-fine-tuned 2-3B small language models outperform base versions and Gemini 2.5 Flash on de-escalation civilian response generation.

Problem

Training data for realistic, edge-deployable de-escalation simulators for police training is scarce; LLMs are too heavy for portable hardware and existing dialogue data lacks tactical de-escalation content.

Approach

Builds a de-escalation dialogue corpus from publicly available social-media police-civilian videos using Whisper, an LLM-based 30-signal filter with human verification, Gemini 2.5 Flash diarization, and NER/LLM anonymization, then fine-tunes small instruct models with QLoRA to generate civilian responses.

Key results

• Fine-tuned Qwen 2.5 (3B) reaches ROUGE-L 15.7, BLEU-4 3.7, METEOR 19.4, BERTScore 0.88 and Realism 62.1/60.8, outperforming Gemini 2.5 Flash (12.4 ROUGE-L, 55.4/54.1 realism).
• Fine-tuned SLMs run at 0.30–0.44 s latency versus 3.50 s for Gemini 2.5 Flash.
• Realism gains from fine-tuning range +18.5 to +28.1 points across models; human-LLM judge correlation rho = 0.81.

Surprising findings

• Fine-tuned Qwen 2.5 (3B) outperforms Gemini 2.5 Flash few-shot on all automatic metrics and both realism judges despite being much smaller.
• Gemma 2 fine-tuned has lower realism scores than Gemini 2.5 Flash, despite surpassing it on automatic metrics.

Limitations

• Benchmark measures linguistic and behavioral plausibility, not actual training efficacy for officers; not validated for field deployment.
• Subjective de-escalation annotation yields inter-annotator variance of 0.14 to 1.9.
• Noisy in-the-wild data with diarization error (DER 6.26-8.16%) and transcription challenges.

Experimental setup

Datasets: DeEscalWild (1,350 training interactions) Social-media police-civilian videos (5,000 raw, 1,500 retained)

Benchmarks: DeEscalWild held-out benchmark (150 scenarios)

Metrics: ROUGE-L BLEU-4 METEOR BERTScore (F1) Realism Score latency De-escalation Rate

Baselines: Gemini 2.5 Flash Gemma 2 (2B-Instruct) Llama 3.2 (3B-Instruct) Granite 3.0 (2B-Instruct) Falcon 3 (3B-Instruct)

Builds on: QLoRA Whisper pyannote LLM-as-a-Judge

Closest papers by meaning

• On Calibration of LLM-based Guard Models for Reliable Content Moderation (ICLR 2025 · similarity 0.84)
• Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.84)
• WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models (NeurIPS 2024 · similarity 0.84)
• WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training (ICML 2025 · similarity 0.83)
• EvoLM: In Search of Lost Language Model Training Dynamics (NeurIPS 2025 · similarity 0.83)
• It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs (NeurIPS 2026 · similarity 0.83)