Research map / Vision-language and multimodal / Visual question answering
Getting Closer to AI Complete Question Answering: A Set of Prerequisite Real Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| evaluate TriAN on QuAIL instead of MCScript | TriAN · QuAIL vs MCScript | accuracy (%): 84% -> 54.7 | Hurt |
| evaluate BERT on QuAIL instead of SWAG | BERT-base · QuAIL vs SWAG | accuracy (%): 86.3% -> 55.9% | Hurt |
| adversarial paraphrasing of questions and answers (ablation) | BERT-base · QuAIL fiction | accuracy: -0.17 | Hurt |
| adversarial paraphrasing of questions and answers (ablation) | TriAN · QuAIL fiction | accuracy: -0.11 | Hurt |
| adversarial paraphrasing of questions and answers (ablation) | PMI and IR · QuAIL fiction | accuracy: -0.03 (PMI), -0.02 (IR) | No clear effect |
| paraphrased data effect on human agreement (ablation) | human volunteers · QuAIL fiction | Krippendorff's alpha: 0.60 -> 0.61 | No clear effect |
| split questions into text+unanswerable vs world knowledge instead of all-in-one (ablation) | human evaluation · QuAIL | Krippendorff's alpha: 0.60 -> 0.78 (text+unanswerable), 0.70 (world knowledge) | Helped |
| use BERT pretraining vs ConceptNet-based TriAN for world knowledge questions | QA baselines · QuAIL world knowledge questions | accuracy (%): TriAN 56.9 vs BERT 63.8 (event duration); TriAN 45.8 vs BERT 48.8 (properties) | Mixed |
| news domain vs other domains for BERT unanswerable questions | BERT-base · QuAIL | accuracy (%) on unanswerable: Blogs 40.0 vs News 68.3 | Mixed |
Closest papers by meaning
- QASA: Advanced Question Answering on Scientific Articles (ICML 2023 · similarity 0.89)
- QASC: A Dataset for Question Answering via Sentence Composition (AAAI 2020 · similarity 0.88)
- Question Answering as Global Reasoning Over Semantic Abstractions (AAAI 2018 · similarity 0.88)
- Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph (AAAI 2018 · similarity 0.88)
- FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering (AAAI 2024 · similarity 0.86)
- Tree-of-Reasoning Question Decomposition for Complex Question Answering with Large Language Models (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.