AI Paper Atlas

Research map / Large language models / AI and society

An Oral Exam for Measuring a Dialog System’s Capabilities

David Cohen, Ian Lane

AAAI 2016 · Large language models › AI and society · approach: Data commons and human baselines · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
compare Human vs Google Noworal exam · Dialog capability oral examtotal performance: 1.04 -> 3.63Improved
compare Google Now vs Cleverbotoral exam · Dialog capability oral examtotal performance: 0.31 -> 1.04Improved
compare Google Now vs Cleverbot on Physical Object Ontologyoral exam · Dialog capability oral examperformance c1: 0.17 -> 0.39Improved
compare Google Now vs Cleverbot on breadthoral exam · Dialog capability oral exambreadth: 0.71 -> 0.91No clear effect
compare Google Now vs Cleverbot on flexibility c1oral exam · Dialog capability oral examflexibility c1: 0.54 -> 0.48No clear effect
compare Google Now vs Cleverbot on flexibility c2oral exam · Dialog capability oral examflexibility c2: 0 -> 0.59No clear effect
compare Human vs Google Now on flexibilityoral exam · Dialog capability oral examflexibility c4: 0.65 -> 1Improved
compare Human vs Google Now on breadthoral exam · Dialog capability oral exambreadth: 0.91 -> 1Improved
Google Now specialized Get Definition vs similar questionsGoogle Now · Dialog capability oral examcapability performanceMixed
use groups of three judges for entropy metricsbreadth and flexibility · Dialog capability oral examreliabilityMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy