Research map / Large language models / Language, safety and interpretability
Vietnamese Words Are Not Constructed from Syllables: Rethinking the Role of Word Segmentation in Natural Language Processing for Vietnamese Texts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper argues from Vietnamese linguistic theory that word segmentation is unnecessary for Vietnamese NLU, and proposes ViWordFormer, a Transformer variant with a phrasal-lexeme score and co-text module that outperforms baselines on most sentence classification and sequence labeling datasets.
Problem
Vietnamese NLP assumes words are built from syllables and relies on word segmentation, but the linguistic basis for this assumption is unclear and its necessity for text understanding is untested.
Approach
Adopts the word-based linguistic view (words = morphemes = syllables; phrasal lexemes as the unit) and designs ViWordFormer, whose attention scores are multiplied by a phrasal score derived from bilinear neighbor-pair kernels, plus a co-text module that propagates phrasal information across layers.
Key results
- ViWordFormer achieves 88.70 macro F1 on ViOCD versus 86.33 for the Transformer baseline, and 84.61 on PhoNER versus 83.37.
- Averaged over datasets, baselines trained on word-segmented input (VnCoreNLP) do not significantly outperform those trained on space-split syllables, e.g. Transformer average 82.54 without segmentation vs 81.20 with segmentation on the two sequence tasks' averages in Table 1.
- Ablation removing the co-text module drops PhoNER F1 from 84.61 to 62.91, and ViWordFormer loses on UIT-ViCTSD Toxic Detection to the Transformer (64.38 vs 67.30).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ViWordFormer instead of Transformer | Vietnamese sentence classification · UIT-VSFC Topic Classification | F1-score: 76.22 -> 77.37 | Helped |
| use ViWordFormer instead of Transformer | Vietnamese sentence classification · UIT-VSFC Sentiment Analysis | F1-score: 75.12 -> 76.92 | Helped |
| use ViWordFormer instead of LSTM | Vietnamese toxic detection · UIT-ViCTSD Toxic Detection | F1-score: 67,79 -> 64.38 | Hurt |
| use ViWordFormer instead of Transformer | Vietnamese sequential labeling · PhoNER | F1-score: 83.37 -> 84.61 | Helped |
| use word-based preprocessing without word segmentation instead of morpheme-based preprocessing with segmentation | Vietnamese NLP baselines · Vietnamese text understanding datasets | F1-score | No clear effect |
| add the Co-text module (ablation) | ViWordFormer · UIT-VSFC Topic Classification | F1-score: 75.60 -> 77,37 | Helped |
| add the Co-text module (ablation) | ViWordFormer · UIT-VSFC Sentiment Analysis | F1-score: 76.53 -> 76,92 | Helped |
| add the Co-text module (ablation) | ViWordFormer · UIT-ViCTSD Toxic Detection | F1-score: 62.22 -> 64,38 | Helped |
| add the Co-text module (ablation) | ViWordFormer · UIT-ViCTSD Constructive Detection | F1-score: 76.01 -> 78,07 | Helped |
| add the Co-text module (ablation) | ViWordFormer · ViHSD | F1-score: 59.87 -> 63,68 | Helped |
| add the Co-text module (ablation) | ViWordFormer · ViOCD | F1-score: 86.33 -> 88,70 | Helped |
| add the Co-text module (ablation) | ViWordFormer · ViHOS | F1-score: 78.00 -> 79,22 | Helped |
Surprising findings
- Word segmentation with VnCoreNLP does not improve, and often slightly lowers, baseline performance across most Vietnamese datasets.
- ViWordFormer underperforms the vanilla Transformer by a large margin on UIT-ViCTSD Toxic Detection.
Limitations
- The phrasal score assumes a linear (bilinear) semantic structure, which may not capture Vietnamese phrasal relations well.
- Experiments are limited to sentence classification and sequence labeling; no generation or reading comprehension tasks were tested.
- Baselines are small models trained from scratch rather than pretrained Vietnamese language models.
Experimental setup
Datasets: UIT-VSFC UIT-ViCTSD UIT-ViOCD ViHSD PhoNER ViHOS
Benchmarks: UIT-VSFC UIT-ViCTSD ViHSD UIT-ViOCD ViHOS PhoNER
Metrics: macro F1
Baselines: Transformer encoder LSTM GRU TextCNN
Builds on: Transformer PhoBERT VnCoreNLP
Closest papers by meaning
- ViDia2Std: A Parallel Corpus and Methods for Low-Resource Vietnamese Dialect-to-Standard Translation (AAAI 2026 · similarity 0.85)
- Fairness in Representation for Multilingual NLP: Insights from Controlled Experiments on Conditional Language Modeling (ICLR 2022 · similarity 0.84)
- Revitalizing Bahnaric Language through Neural Machine Translation: Challenges, Strategies, and Promising Outcomes (AAAI 2024 · similarity 0.84)
- A Latent Morphology Model for Open-Vocabulary Neural Machine Translation (ICLR 2020 · similarity 0.83)
- The Foundations of Tokenization: Statistical and Computational Concerns (ICLR 2025 · similarity 0.81)
- From Tokens to Words: On the Inner Lexicon of LLMs (ICLR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.