Research map / Large language models / Language, safety and interpretability

ViFactCheck: A New Benchmark Dataset and Methods for Multi-Domain News Fact-Checking In Vietnamese

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Text generation and evaluation metrics · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ViFactCheck, a 7,232-sample human-annotated Vietnamese multi-domain news claim-evidence fact-checking benchmark, and benchmarks fine-tuned PLMs/LLMs and prompted LLMs on it, with fine-tuned Gemma reaching 89.90 macro F1 with gold evidence.

Problem

Fact-checking resources are scarce for low-resource languages such as Vietnamese, and existing Vietnamese fact-checking data (e.g., ViWikiFC) is Wikipedia-based and single-evidence, not multi-domain news.

Approach

A human-curated Vietnamese news fact-checking dataset where annotators write Support/Refute/NEI claims from article contexts and mark multi-sentence evidence spans, rather than FEVER-style automated claim mutation; benchmarked with fine-tuning (LoRA on LLMs) and zero-shot prompting.

Key results

• Fine-tuned Gemma achieved the best macro F1 of 89.90 with gold evidence and 85.94 with full context on the ViFactCheck test set.
• Fine-tuned XLM-R_large reached 88.02 macro F1 with gold evidence, the best PLM result.
• Prompted LLMs performed far below fine-tuning; prompted Gemini reached 76.26 with full context, and prompted Gemma only 45.05.

Measured effects reported in the paper

Surprising findings

• Prompted LLMs (e.g., Gemma, Llama2) performed much worse than fine-tuned counterparts, and Llama2 fine-tuning jumped from 41.47 with full context to 79.53 with gold evidence.
• Human annotators scored 84.93 macro F1 on a 200-sample subset, below fine-tuned Gemma (83.95) and Llama3 (84.24) on that subset only marginally, while humans were weaker on Refute claims.

Limitations

• Relatively small dataset (7,232 samples) and a small Vietnamese news corpus of 1,000 articles from February-March 2023.
• Evidence retrieval and multi-evidence reasoning remain challenging; error analysis highlights semantic ambiguity and complex inferential chains.

Experimental setup

Datasets: ViFactCheck

Benchmarks: ViFactCheck

Metrics: macro F1 Fleiss kappa per-class F1

Baselines: PhoBERT ViBERT mBERT XLM-R Llama2 Llama3 Mistral Gemini 1.5 Flash

Builds on: FEVER ViWikiFC PhoBERT XLM-R

Closest papers by meaning

• CFEVER: A Chinese Fact Extraction and VERification Dataset (AAAI 2024 · similarity 0.84)
• Pioneering Explainable Video Fact-Checking with a New Dataset and Multi-role Multimodal Model Approach (AAAI 2025 · similarity 0.84)
• The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality (NeurIPS 2026 · similarity 0.84)
• Drifting Away from Truth: GenAI-Driven News Diversity Challenges LVLM-Based Misinformation Detection (AAAI 2026 · similarity 0.84)
• Long-form factuality in large language models (NeurIPS 2024 · similarity 0.83)
• Contradicted in Reliable, Replicated in Unreliable: Dual-Source Reference for Fake News Early Detection (AAAI 2025 · similarity 0.83)