Research map / Vision-language and multimodal / Visual question answering
An Empirical Study of Content Understanding in Conversational Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Mask previous answer text while retaining its position, rather than removing all conversation history (ablation) | QuAC models · QuAC validation | F1: 54.1 (.13) -> 62.4 (.20); 55.3 (.03) -> 62.2 (.48) | Helped |
| Mask previous answer text while retaining its position, rather than removing all conversation history (ablation) | CoQA models · CoQA validation | F1: 63.4 (.11) -> 71.5 (.24); 68.3 (.45) -> 74.0 (.19) | Helped |
| Repeat answer-containing passage text between answer spans at test time (ablation) | FlowQA and BERT · QuAC validation | F1: 64.4 (0.30) -> 53.3 (0.75); 63.6 (0.16) -> 55.3 (0.74) | Hurt |
| Train without previous-answer position information (ablation) | FlowQA and BERT · QuAC validation under repeat attack | F1: 56.9 (0.37) vs 53.3 (0.75); 56.5 (0.22) vs 55.3 (0.74) | Helped |
| Repeat answer-containing passage text between answer spans at test time (ablation) | FlowQA and SDNet · CoQA validation | F1: 76.9 (0.22) -> 72.0 (0.20); 76.4 (0.31) -> 71.8 (0.48) | Hurt |
| Mask previous answer text at inference (ablation) | FlowQA · QuAC validation | F1: 64.4 (0.30) -> 60.5 (0.54) | Hurt |
| Mask previous answer text at inference (ablation) | BERT · QuAC validation | F1: 63.6 (0.16) -> 52.6 (1.76) | Hurt |
| Remove previous-answer position information at inference (ablation) | FlowQA · QuAC validation | F1: 64.4 (0.30) -> 48.1 (0.72) | Hurt |
| Remove previous-answer position information at inference (ablation) | BERT · QuAC validation | F1: 63.6 (0.16) -> 54.9 (0.08) | Hurt |
| Remove previous-answer position information at inference (ablation) | FlowQA · CoQA validation | F1: 76.9 (0.22) -> 76.7 (0.36) | No clear effect |
| Shuffle passage sentences (ablation) | FlowQA - position · CoQA validation | F1: 76.8 (0.42) -> 71.7 (0.25) | Hurt |
| Shuffle passage sentences (ablation) | SDNet - position · CoQA validation | F1: 75.7 (0.76) -> 70.1 (0.59) | Hurt |
Closest papers by meaning
- SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs (ICLR 2024 · similarity 0.88)
- RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content (NeurIPS 2024 · similarity 0.88)
- ChatQA: Surpassing GPT-4 on Conversational QA and RAG (NeurIPS 2024 · similarity 0.88)
- A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering (NeurIPS 2026 · similarity 0.88)
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.87)
- Rethinking Generative Large Language Model Evaluation for Semantic Comprehension (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.