Research map / Vision-language and multimodal / Visual question answering
MMClima: A Framework for Multimodal Climate Science Data and Evaluation
Muhammad Umer Sheikh, Hassan Abid, Khawar shehzad, Ufaq Khan, Muhammad Haris Khan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Llama 3.3 70B with LoRA on MMClima training split | Llama-3.3-70B-Instruct · MMClima textual QA | weighted overall score: 71.42 -> 78.24 | Improved |
| fine-tune Llama 3.3 70B on MMClima | Llama-3.3-70B-Instruct · MMClima textual QA MCQ | accuracy: 80.26 -> 89.54 | Improved |
| fine-tune Llama 3.3 70B on MMClima | Llama-3.3-70B-Instruct · MMClima textual QA Cloze | score: 45.56 -> 49.72 | Improved |
| fine-tune Llama 3.3 70B on MMClima | Llama-3.3-70B-Instruct · MMClima textual QA Freeform | BERTScore: 88.43 -> 95.47 | Improved |
| fine-tune Llama 3.3 70B on MMClima | Llama-3.3-70B-Instruct · MMClima freeform QA | LLM judge accuracy: 45.48 -> 52.29 | Improved |
| use YouTube transcript questions instead of Wikipedia text questions (ablation) | mmclima_70b_txt · MMClima textual QA | overall (unweighted mean): 78.08 -> 80.17 | Improved |
| use YouTube transcript questions instead of Wikipedia text questions (ablation) | mistral-7b-instruct · MMClima textual QA MCQ | accuracy: 32.62 -> 52.26 | Improved |
| use YouTube transcript questions instead of Wikipedia text questions (ablation) | gpt-5-nano · MMClima textual QA Freeform | accuracy: 87.78 -> 88.33 | No clear effect |
| use closed-source GPT-5 instead of open-source Pixtral-Large | VLM · MMClima VQA | weighted overall score: 68.53 -> 71.94 | Improved |
Closest papers by meaning
- ClimaQA: An Automated Evaluation Framework for Climate Question Answering Models (ICLR 2025 · similarity 0.92)
- Assessing Large Language Models on Climate Information (ICML 2024 · similarity 0.90)
- SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers (NeurIPS 2024 · similarity 0.86)
- MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.86)
- GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensing (CVPR 2026 · similarity 0.86)
- UPME: An Unsupervised Peer Review Framework for Multimodal Large Language Model Evaluation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
