Research map / Label-efficient and robust learning / Recognition and long-tail
Improving Unsupervised Constituency Parsing via Maximizing Semantic Information
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes maximizing SemInfo (semantic information in constituent structures, estimated via LLM paraphrases, a bag-of-substrings model and PWI) as an objective for PCFG-based unsupervised constituency parsing, trained via a TreeCRF with REINFORCE.
Problem
Unsupervised constituency parsers trained by maximizing sentence log-likelihood show weak correlation between LL and parsing accuracy, so LL is an insufficient objective for learning syntax that reflects semantics.
Approach
Estimate semantic information of constituent substrings with a bag-of-substrings model built from LLM paraphrases, applying the probability-weighted information (tf-idf-like) metric over maximal substrings, and use the summed SemInfo of a tree as a reinforcement-learning reward for training PCFGs through a TreeCRF with mean-field optimization.
Key results
- SemInfo-trained PCFGs improve SF1^c over LL-trained PCFGs by an average of 13.09 (English), 6.02 (Chinese), 7.31 (French), and 4.92 (German) across five PCFG variants.
- SemInfo correlates with sentence-level SF1 at roughly 0.6-0.9 (Spearman), while LL correlates near 0.
- SemInfo-trained PCFGs reach state-of-the-art-level non-ensemble parsing, outperforming the second-best method by 1.82 (English), 11.02 (Chinese), and 4.47 (French) SF1^c.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace likelihood maximization with SemInfo maximization | PCFG variants · English, Chinese, French, and German | SF1^c: English +13.09; Chinese +6.02; French +7.31; German +4.92 | Helped |
| Replace likelihood maximization with SemInfo maximization | PCFG parsers · Five PCFG variants across four languages | SF1^c: 7.85 sentence-F1 scores | Helped |
| Use SemInfo rather than LL to score predicted parses | PCFG parsers · English validation | Spearman correlation with sentence SF1^i: 0.6347 vs -0.0074 | Helped |
| Use SemInfo-trained PCFG instead of MaxTreeDecoding | Unsupervised constituency parsing · English, Chinese, and French | SF1^c: 1.82, 11.02, and 4.47 SF1^c scores | Helped |
| Use SemInfo-trained PCFG instead of MaxTreeDecoding | Unsupervised constituency parsing · German | SF1^c | Hurt |
| Replace TreeCRF-based optimization with PCFG-based optimization (ablation) | SemInfo-trained NPCFG and SNPCFG · English | SF1^c | No clear effect |
| Use PCFG-based Action-V optimization instead of TreeCRF-based Action-V (ablation) | NPCFG · English | SF1^c: 65.16±1.15 -> 62.66±1.76 | Hurt |
| Replace the sample-average baseline with a V-function baseline (ablation) | TreeCRF-based Posterior optimization · Parsing | SF1^c | Helped |
| Use log-normalized rather than unnormalized maximal-substring frequency (ablation) | SemInfo · Parsing | SF1^c | Helped |
| Use a smaller or noisier paraphrasing model (ablation) | SemInfo-trained NPCFG · English | SF1^c: 65.19±0.54 -> 59.01±0.24 | Hurt |
| Train SemInfo-NPCFG with qwen2.5-0.5b paraphrases instead of LL training (ablation) | NPCFG · English | SF1^c: 59.01±0.24 vs 50.96±1.82 | Helped |
| Use TreeCRF-based optimization instead of direct PCFG optimization (ablation) | NPCFG · Training efficiency | speed: 4x | Helped |
Surprising findings
- LL correlates strongly with corpus-level parsing accuracy early in training but this correlation drops below 0.4 later, while SemInfo stays strong.
- The GPT-4o-mini direct parser performs far worse than the maximum-tree-decoding parser, especially in Chinese (11.82 vs 49.03).
Limitations
- German results lag behind the maximum tree decoding baseline, attributed to vocabulary shift between training and test sets.
- Method is demonstrated only on unlexicalized PCFGs; applicability to lexicalized PCFGs left to future work.
- Performance depends on paraphrasing LLM quality, with small paraphrasers yielding lower accuracy.
Experimental setup
Datasets: Penn TreeBank Chinese Treebank 5.1 SPMRL
Benchmarks: Penn TreeBank Chinese Treebank 5.1 SPMRL
Metrics: corpus-level sentence F1 (SF1^c) instance-level sentence F1 (SF1^i) Spearman correlation
Baselines: MaxTreeDecoding GPT4o-mini parser Spanoverlap SCPCFG (2048NT) SNPCFG (4096NT) URNNG NBL-PCFG S-DIORA Constituency Test
Builds on: PCFG Compound PCFG Neural PCFG tf-idf / PWI TreeCRF
Closest papers by meaning
- Ensemble Distillation for Unsupervised Constituency Parsing (ICLR 2024 · similarity 0.89)
- Deep networks learn to parse uniform-depth context-free languages from local statistics (ICML 2026 · similarity 0.88)
- Improving Sequence-to-Sequence Constituency Parsing (AAAI 2018 · similarity 0.87)
- BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing (NeurIPS 2023 · similarity 0.87)
- Dependency Parsing is More Parameter-Efficient with Normalization (NeurIPS 2025 · similarity 0.85)
- Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.