Research map / Large language models / Reasoning and chain of thought

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering

AAAI 2024 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 37 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes T-SciQ, which uses LLM-generated CoT and plan-based CoT rationales (mixed per skill via validation) as teaching signals to fine-tune small multimodal students, reaching 96.18% on ScienceQA.

Problem

Human-annotated CoT rationales for multimodal science question answering are costly to collect and often miss essential external knowledge, limiting small multimodal models' CoT reasoning on ScienceQA.

Approach

Generate teaching rationales from an LLM (text-davinci-003) using a hint-conditioned zero-shot CoT prompt and a 3-step plan-based CoT prompt (lecture, plan, rationale), then pick per skill, using validation accuracy, whether the CoT or PCoT signal is used to fine-tune a two-stage Multimodal-CoT student.

Key results

• Mutimodal-T-SciQ Large reaches 96.18% average accuracy on ScienceQA test, versus 91.68% for Mutimodal-CoT Large trained on annotated CoT.
• Mutimodal-T-SciQ Base reaches 91.75% vs 84.91% for Mutimodal-CoT Base.
• On six NLP reasoning datasets, T-SciQ outperforms Reason-Teacher on 5 of 6 and ties on Coin Flip (98.67%).

Measured effects reported in the paper

Surprising findings

• LLM-generated rationales from text-davinci-003 outperform human-annotated CoT rationales even when the mixing uses only QA-CoT signals, which the authors attribute to annotation redundancy and limited external knowledge.
• Even the weakest teacher mixture outperforms annotated CoT signals by a significant margin.

Limitations

• Relies on proprietary GPT-3.5 (text-davinci-003) teacher and API-based generation, so reproducibility depends on API access.
• PCoT rationales may be erroneous when visual features are complicated, per the appendix.
• Mixing selects one rationale type per skill using validation accuracy, a coarse granularity.

Experimental setup

Datasets: ScienceQA Aqua Coin Flip CommonSenseQA StrategyQA Date Understanding Tracking Shuffled Objects

Benchmarks: ScienceQA Aqua Coin Flip CommonSenseQA StrategyQA Date Understanding Tracking Shuffled Objects

Metrics: accuracy

Baselines: Multimodal-CoT UnifiedQA LLaVA LLaMA-Adapter Chameleon GPT-4 w/ CoT ChatGPT w/ CoT GPT-3.5 w/ CoT Reason-Teacher MCAN Top-Down BAN

Builds on: Multimodal-CoT chain-of-thought prompting plan-and-solve prompting Reason-Teacher Fine-tune-CoT

Closest papers by meaning

• KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning (AAAI 2024 · similarity 0.90)
• MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.89)
• MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.89)
• Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs (NeurIPS 2024 · similarity 0.88)
• CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs (ICLR 2026 · similarity 0.87)
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning (NeurIPS 2024 · similarity 0.87)