Research map / Large language models / Agents, code and math
ECLAIR: Enhanced Clarification for Interactive Responses in an Enterprise AI Assistant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ECLAIR, a multi-agent framework for interactive disambiguation of enterprise AI assistant queries, evaluated on in-house annotated Adobe Experience Platform user queries against a GPT-3.5 few-shot baseline.
Problem
LLMs struggle to resolve ambiguous user queries in real-world enterprise interactions, where context and domain-specific knowledge matter, and existing sequential detect-then-clarify pipelines and synthetic datasets fail to capture this.
Approach
A modular multi-agent framework where developer-defined ambiguity detection and grounding agents (sentence-level, product, entity linking, concept graph) produce signals that are fed into an LLM prompt to decide whether to ask a clarification question, which is then refined with user feedback.
Key results
- ECLAIR reaches average F1 of 0.657 versus 0.520 for the few-shot GPT-3.5 baseline on clarification-needed decisions.
- ECLAIR achieves higher precision on 'Clarification Needed' (0.904 vs 0.732) but lower recall (0.635 vs 0.833).
- The paper reports ECLAIR's overall F1 is 13% higher than the few-shot method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Clar. Needed | precision: 0.732 -> 0.904 | Helped |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Clar. Needed | recall: 0.833 -> 0.635 | Hurt |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Clar. Needed | F1: 0.779 -> 0.746 | Hurt |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Clar. Not Needed | precision: 0.333 -> 0.438 | Helped |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Clar. Not Needed | recall: 0.214 -> 0.808 | Helped |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Clar. Not Needed | F1: 0.261 -> 0.568 | Helped |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Avg | precision: 0.533 -> 0.671 | Helped |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Avg | recall: 0.524 -> 0.721 | Helped |
| Use ECLAIR instead of the few-shot baseline | GPT-3.5 clarification decision · AEP ambiguity detection, Avg | F1: 0.520 -> 0.657 | Helped |
Surprising findings
- ECLAIR has lower recall and F1 than the baseline on the 'Clarification Needed' class, trading recall for precision.
Limitations
- Evaluation is preliminary and limited to a binary clarification-needed decision on an in-house dataset.
- Dataset is proprietary and not released, limiting reproducibility.
- Only GPT-3.5 is evaluated as the backbone.
Experimental setup
Datasets: Adobe Experience Platform AI Assistant user queries (in-house annotated)
Benchmarks: in-house AEP user query evaluation set
Metrics: precision recall F1
Baselines: few-shot GPT-3.5 prompting
Builds on: chain-of-thought prompting
Closest papers by meaning
- ECLAIR: Enhanced Clarification for Interactive Responses (AAAI 2025 · similarity 0.97)
- Uncertainty-Aware Clarification in LLM Agents with Information Gain (ICML 2026 · similarity 0.89)
- InteractComp: Evaluating Search Agents With Ambiguous Queries (ICML 2026 · similarity 0.88)
- Active Task Disambiguation with LLMs (ICLR 2025 · similarity 0.86)
- Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.86)
- Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.