Research map / Large language models / Agents, code and math
Do Large Language Models Reason About Uncertainty Like Humans? A Benchmark on Hurricane Forecast Visualization Comprehension
Le Liu, Yuhao Wang, Bohan Shen, Wei Zeng, Shizhou Zhang, Di Xu, Peng Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add conversational context to GPT-4o (ablation) | GPT-4o · Phase 1 central damage rating | damage rating (beta): beta = -0.32 | Mixed |
| add conversational context to GPT-4o (ablation) | GPT-4o · Phase 1 distance effect, cone-centerline | slope vs distance: -0.045 -> 0.021 | Mixed |
| add annotations of storm size and intensity to ensemble display (ablation) | humans and GPT-4o · Phase 2 distance effect | slope vs distance: human coef. 0.037; LLM w/ ctx 0.024; w/o ctx 0.012 | Improved |
| add annotations to representative-ensemble (ablation) | GPT-4o · Phase 2 center rating | damage rating (beta): humans -0.908; LLM w/o ctx +0.358; w/ ctx +0.938 | Mixed |
| add context to GPT-4o (ablation) | GPT-4o · Phase 2 time x center effect | time effect alignment with humans | Improved |
| add context to GPT-4o (ablation) | GPT-4o · Phase 1 cone-only time effect | time x center beta: beta = -0.501 | Improved |
| use ensemble (implicit) instead of explicit cone display (ablation) | GPT-4o without context · Phase 1 time x center | 48h vs 24h rating: ensemble beta = -1.300 vs cone-centerline +0.613 | Improved |
| add context to GPT-4o (ablation) | GPT-4o · Phase 1 confidence | mean confidence (1-7): 5 -> 6 | Mixed |
| add context to GPT-4o (ablation) | GPT-4o · reasoning strategy distribution | strategy usage | No clear effect |
| add prompt clarification separating uncertainty from severity (ablation) | GPT-4o · cone width misconception | bias | Improved |
Closest papers by meaning
- Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models (ICLR 2026 · similarity 0.85)
- CyPortQA: Benchmarking Multimodal Large Language Models for Cyclone Preparedness in Port Operation (AAAI 2026 · similarity 0.85)
- ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.85)
- Assessing Large Language Models on Climate Information (ICML 2024 · similarity 0.84)
- PlanU: Large Language Model Reasoning through Planning under Uncertainty (NeurIPS 2025 · similarity 0.84)
- Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
