Research map / Large language models / Language, safety and interpretability
Divide-Solve-Combine: An Interpretable and Accurate Prompting Framework for Zero-shot Multi-Intent Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DSCP, a three-stage prompting framework (divide, solve intent-by-intent, combine) for zero-shot multi-intent detection, plus an interactive multi-turn variant Inter-DSCP, evaluated on MIXATIS and MIXSNIPS with several LLMs.
Problem
Zero-shot multi-intent detection with LLMs suffers from a lack of explicit reasoning and interpretability in existing prompting approaches.
Approach
Decompose multi-intent detection into single-intent span division, per-span intent solving, and final combination via sequential prompts; the interactive variant spreads these steps across three dialogue turns to exploit LLM multi-turn interaction.
Key results
- On MIXSNIPS with GPT-4, Inter-DSCP reaches 92.00% intent accuracy vs 87.77% for Zero-CoT and 88.45% for VMID.
- On MIXATIS with GPT-4, Least-to-Most reaches 48.33% intent accuracy while DSCP reaches 50.69% and Inter-DSCP 52.22%.
- With GPT-3.5 on MIXATIS, Inter-DSCP improves intent accuracy to 41.11% from 26.11% for Zero-CoT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DSCP instead of VMID | GPT-4 · MIXSNIPS | Intent Acc.: 88.45 -> 89.68 | Helped |
| Use Inter-DSCP instead of DSCP | GPT-4 · MIXSNIPS | Intent Acc.: 89.68 -> 92.00 | Helped |
| Use DSCP instead of the strongest baseline, Least-to-Most | GPT-4 · MIXATIS | Intent Acc.: 48.33 -> 50.69 | Helped |
| Use Inter-DSCP instead of DSCP | GPT-4 · MIXATIS | Intent Acc.: 50.69 -> 52.22 | Helped |
| Add single-intent division prompting (ablation) | DSCP with GPT-3.5 · MIXSNIPS | Intent Acc.: 70.44 -> 72.90 | Helped |
| Add intent-by-intent solution prompting (ablation) | DSCP with GPT-3.5 · MIXSNIPS | Intent Acc.: 67.62 -> 72.90 | Helped |
| Add multi-intent combination prompting (ablation) | DSCP with GPT-3.5 · MIXSNIPS | Intent Acc.: 66.07 -> 72.90 | Helped |
| Add single-intent division prompting (ablation) | DSCP with GPT-3.5 · MIXATIS | Intent Acc.: 13.61 -> 29.17 | Helped |
| Add intent-by-intent solution prompting (ablation) | DSCP with GPT-3.5 · MIXATIS | Intent Acc.: 24.72 -> 29.17 | Helped |
| Add multi-intent combination prompting (ablation) | DSCP with GPT-3.5 · MIXATIS | Intent Acc.: 12.64 -> 29.17 | Helped |
Surprising findings
- Zero-shot chain-of-thought ('Let's think step by step!') often fails to induce step-by-step reasoning in multi-intent tasks, directly giving answers instead.
- With GPT-3.5, three-shot DSCP prompting on MIXATIS reportedly exceeds GPT-4 zero-shot performance.
Limitations
- Evaluated only on two English benchmarks (MIXATIS and MIXSNIPS).
- Interpretability assessed by manual annotator scores rather than a formal metric.
- Mistral-7B results are much lower than GPT-family models, and DSCP does not always beat baselines on Mistral-7B (e.g., Zero-CoT has higher Macro F1).
Experimental setup
Datasets: MIXATIS MIXSNIPS
Benchmarks: MIXATIS MIXSNIPS
Metrics: Intent Accuracy Macro F1 Micro F1
Baselines: VMID Zero-CoT Least-to-Most Plan-and-Solve
Builds on: Chain-of-thought prompting Least-to-Most prompting Plan-and-Solve prompting VMID
Closest papers by meaning
- Uni-MIS: United Multiple Intent Spoken Language Understanding via Multi-View Intent-Slot Interaction (AAAI 2024 · similarity 0.87)
- Towards Multi-Intent Spoken Language Understanding via Hierarchical Attention and Optimal Transport (AAAI 2024 · similarity 0.87)
- MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations (ICLR 2024 · similarity 0.87)
- Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning (AAAI 2024 · similarity 0.86)
- Aligner²: Enhancing Joint Multiple Intent Detection and Slot Filling via Adjustive and Forced Cross-Task Alignment (AAAI 2024 · similarity 0.86)
- Instance-adaptive Zero-shot Chain-of-Thought Prompting (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.