Research map / Vision-language and multimodal / Visual question answering
Zero-Shot Commonsense Question Answering with Cloze Translation and Consistency Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Translate natural questions into cloze form with syntactic rules | ALBERT-xxlarge-v2 · CommonsenseQA | accuracy (%): 31.14 -> 50.94 (dev); 28.52 -> 48.67 (test) | Helped |
| Translate natural questions into cloze form with syntactic rules | ALBERT-xxlarge-v2 · OpenbookQA | accuracy (%): 31.80 -> 41.60 (dev); 33.00 -> 39.80 (test) | Helped |
| Translate natural questions into cloze form with syntactic rules | ALBERT-xxlarge-v2 · SocialIQA | accuracy (%): 41.71 -> 44.11 (dev); 40.47 -> 42.00 (test) | Helped |
| Add consistency optimization across different cloze translations using unlabeled data | Ensemble of cloze translation predictions · CommonsenseQA | accuracy (%): 54.62 -> 64.07 (dev); 51.57 -> 61.08 (test) | Helped |
| Add consistency optimization across different cloze translations using unlabeled data | Ensemble of cloze translation predictions · OpenbookQA | accuracy (%): 41.00 -> 50.27 (dev); 39.20 -> 49.87 (test) | Helped |
| Add consistency optimization across different cloze translations using unlabeled data | Ensemble of cloze translation predictions · SocialIQA | accuracy (%): 44.11 -> 54.13 (dev); 42.04 -> 54.21 (test) | Helped |
| Combine the knowledge-base method with cloze translation and consistency optimization | Knowledge-base CQA model · CommonsenseQA | accuracy (%): 68.63 -> 69.73 (dev); 66.88 -> 67.38 (test) | Helped |
| Combine the knowledge-base method with cloze translation and consistency optimization | Knowledge-base CQA model · OpenbookQA | accuracy (%): 45.40 -> 58.27 (dev); 48.00 -> 54.27 (test) | Helped |
| Combine the knowledge-base method with cloze translation and consistency optimization | Knowledge-base CQA model · SocialIQA | accuracy (%): 56.04 -> 59.85 (dev); 52.28 -> 59.88 (test) | Helped |
| Use unsupervised seq2seq cloze translation | ALBERT-xxlarge-v2 · SocialIQA | accuracy (%): 41.71 -> 40.94 (dev); 40.47 -> 38.80 (test) | Hurt |
| Combine translation methods by direct ensembling rather than consistency optimization | Cloze translation predictions · CommonsenseQA | accuracy (%): 64.07 vs 54.62 (dev); 61.08 vs 51.57 (test) | Hurt |
| Use syntactic cloze translation instead of natural questions | ALBERT-xxlarge-v2 · CommonsenseQA few-shot | accuracy | Helped |
Closest papers by meaning
- FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering (AAAI 2024 · similarity 0.87)
- RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content (NeurIPS 2024 · similarity 0.85)
- Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures (NeurIPS 2026 · similarity 0.85)
- A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering (NeurIPS 2026 · similarity 0.85)
- Multimodal Commonsense Knowledge Distillation for Visual Question Answering (Student Abstract) (AAAI 2025 · similarity 0.85)
- CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Models (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.