Research map / Multi-agent systems and games / LLM agents
ECLAIR: Enhanced Clarification for Interactive Responses
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ECLAIR, a unified single-call prompt framework that integrates outputs from modular ambiguity-detection and grounding agents to decide on and generate clarification questions for enterprise AI assistant queries, evaluated on 100 real AEP AI Assistant queries against a few-shot GPT-3.5 baseline.
Problem
Ambiguous user queries in enterprise AI assistants need clarification, but sequential LLM pipelines detect ambiguity lexically without enterprise context and existing datasets are synthetic and lack real-world complexity.
Approach
A single prompt that fuses ambiguity findings and domain grounding information from multiple pluggable downstream agents (entity linking, product classifier, generic ambiguity, concept graph) so that one LLM call makes the binary ambiguity decision and generates the clarification question.
Key results
- On the binary clarification-needed decision, ECLAIR reaches 0.904 precision on the 'clarification needed' class versus 0.732 for the few-shot baseline, but lower recall (0.635 vs 0.833).
- Overall average F1 is 0.657 for ECLAIR vs 0.520 for the baseline (about 13 points higher).
- BERTScore of generated clarification questions against gold annotations is 0.886 for ECLAIR vs 0.868 for the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Clarification Needed · AEP AI Assistant real user queries | Precision: 0.732 -> 0.904 | Helped |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Clarification Needed · AEP AI Assistant real user queries | Recall: 0.833 -> 0.635 | Hurt |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Clarification Needed · AEP AI Assistant real user queries | F1: 0.779 -> 0.746 | Hurt |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Clarification Not Needed · AEP AI Assistant real user queries | Precision: 0.333 -> 0.438 | Helped |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Clarification Not Needed · AEP AI Assistant real user queries | Recall: 0.214 -> 0.808 | Helped |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Clarification Not Needed · AEP AI Assistant real user queries | F1: 0.261 -> 0.568 | Helped |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Average · AEP AI Assistant real user queries | Precision: 0.533 -> 0.671 | Helped |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Average · AEP AI Assistant real user queries | Recall: 0.524 -> 0.721 | Helped |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification decisions, Average · AEP AI Assistant real user queries | F1: 0.520 -> 0.657 | Helped |
| Use unified ECLAIR prompting instead of the few-shot baseline | AEP AI Assistant clarification question generation · AEP AI Assistant real user queries | BERTScore: 0.868 -> 0.886 | Helped |
Surprising findings
- Annotators agreed strongly on semantics (BERTScore 0.904) but only moderately on wording (ROUGE-L 0.395).
- The baseline's 'clarification needed' F1 (0.779) is higher than ECLAIR's (0.746), so ECLAIR is not better on every metric.
Limitations
- Evaluation uses only 100 hard real user queries from a single proprietary enterprise platform.
- Precision target of 98% for deployment is not reached; clarification-needed recall drops relative to baseline.
- No user study; latency not measured.
Experimental setup
Datasets: AEP AI Assistant real user queries (100 hard cases)
Benchmarks: AEP AI Assistant real user query evaluation set (100 queries)
Metrics: precision recall F1 BERTScore ROUGE-L
Baselines: few-shot prompting baseline with GPT-3.5 and existing ambiguity detector plus qu
Builds on: chain-of-thought prompting CLAMBER BERTScore
Closest papers by meaning
- ECLAIR: Enhanced Clarification for Interactive Responses in an Enterprise AI Assistant (AAAI 2025 · similarity 0.97)
- Uncertainty-Aware Clarification in LLM Agents with Information Gain (ICML 2026 · similarity 0.89)
- InteractComp: Evaluating Search Agents With Ambiguous Queries (ICML 2026 · similarity 0.88)
- Active Task Disambiguation with LLMs (ICLR 2025 · similarity 0.86)
- Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents? (NeurIPS 2026 · similarity 0.86)
- Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.