Research map / Multi-agent systems and games / LLM agents
A Framework for Mining Speech-to-Text Transcripts of the Customer for Automated Problem Remediation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a pipeline that mines speech-to-text transcripts of technical support calls (punctuation BiRNN, GRU keyword extraction, PAS phrase completion, technical-corpus filtering, CRF attribute tagging) to enrich agent-written service requests and improve IR-based problem resolution.
Problem
Support agents' service requests often miss key symptoms, activities, and intents voiced by customers in calls, which degrades downstream search-based automated problem remediation.
Approach
Combine a BiRNN punctuation/capitalization model, GRU-based keyword extraction with attention-style importance gates, PAS-based phrase completion, and technical-corpus n-gram filtering to extract enrichment text from STT transcripts, then use a linear-chain CRF to tag symptom, activity, and intent attributes, feeding the enriched text into search with boosted or penalized relevance.
Key results
- Enrichment detected information missing from agent summaries in 97 of 240 records (about 40%) per manual evaluation.
- CRF attribute extraction achieved F1 of 74.33% for symptoms, 44.85% for activities, and 77.41% for intents.
- SME evaluation showed an overall 55% accuracy improvement in problem resolution when search used enriched information.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Enrich agent service-request summaries with information extracted from STT transcripts | Information retrieval for problem remediation · Problem remediation process | accuracy: 55% | Helped |
| Apply section detection and keyword extraction to speech transcripts | Text processing framework · 240 support calls | similarity: 30% −40% | Helped |
| Use PAS parsing and technical-corpus phrase stitching after keyword extraction | Text processing framework · 240 support calls | similarity: (5-10)% | Helped |
| Extract missing information from STT transcripts and compare it with agent summaries | Text processing framework · 240 service-request records | enrichment detection improvement: 97 out of 240 ... 40% improvement | Helped |
Surprising findings
- Agent summaries captured only 18-20% of reported activities in service requests.
- Enrichment can also expose erroneous agent-added content, such as an unsupported 'windows' term.
Experimental setup
Datasets: 240 STT transcripts from a technical support company 1972 annotated support queries Intelligence Squared transcripts IWSLT 2012 MT transcripts 30 GB technical text dump (manuals, articles, service requests)
Benchmarks: 240 real STT transcripts with agent service requests
Metrics: Cosine similarity Jaccard similarity Jensen-Shannon divergence precision recall F1 kappa coefficient SME-judged search accuracy
Baselines: agent-written service request summaries
Builds on: GRU Predicate Argument Structure parsing linear-chain CRF
Closest papers by meaning
- Talk, Snap, Complain: Validation-Aware Multimodal Expert Framework for Fine-Grained Customer Grievances (AAAI 2026 · similarity 0.79)
- icsPLMs: Exploring Pre-trained Language Models in Intelligent Customer Service (Student Abstract) (AAAI 2024 · similarity 0.78)
- DABL: Detecting Semantic Anomalies in Business Processes Using Large Language Models (AAAI 2025 · similarity 0.78)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.77)
- $\tau$-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge (ICML 2026 · similarity 0.76)
- Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation (NeurIPS 2025 · similarity 0.76)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.