Research map / Large language models / Language, safety and interpretability
Words That Unite The World: A Unified Framework for Deciphering Central Bank Communications
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the WCB dataset of 380k+ sentences from 25 central banks, with 25k annotated sentences across stance, temporal, and uncertainty tasks, and benchmarks PLMs and LLMs under bank-specific and pooled setups.
Problem
Central bank communications are hard to interpret consistently across institutions, and existing monetary-policy NLP resources are small, geographically narrow, and lack annotation guides for stance, temporal orientation and uncertainty.
Approach
A large multi-bank monetary policy corpus with bank-specific annotation guides and three sentence-level labels (stance with an added Irrelevant class, forward-looking, uncertainty), plus a comparison of models trained on pooled data versus per-bank data showing cross-bank pooling helps.
Key results
- RoBERTa-Large achieves the best average weighted F1 of 0.740 on Stance Detection in the General (All-Banks) setup among PLMs.
- Llama-3-70B zero-shot reaches average weighted F1 of 0.620 on Stance Detection, the best LLM, while FinMA is lowest.
- Models trained on pooled data across banks outperform bank-specific models, supporting the 'whole is greater than the sum of its parts' claim (cross-bank similarity correlates with gains, p=0.016).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train RoBERTa-Large on aggregated all-bank data instead of bank-specific data | RoBERTa-Large · WCB Stance Detection | weighted F1 | Helped |
| Increase training sample size up to 600 samples (ablation) | Best-performing stance model · WCB Stance Detection | F1-Score | Helped |
| Increase training sample size beyond 600 (ablation) | Best-performing stance model · WCB Stance Detection | F1-Score: 0.74 | No clear effect |
| Use few-shot prompting instead of zero-shot prompting (ablation) | Llama-3-70B-Chat · WCB Stance Detection | performance | Helped |
| Prompt with the annotation guide instead of zero-shot prompting (ablation) | Llama-3-70B-Chat · WCB Stance Detection | performance | Helped |
| Use few-shot prompting instead of zero-shot prompting (ablation) | Llama-3-70B-Chat · WCB Temporal Classification | performance | Hurt |
| Prompt with the annotation guide instead of zero-shot prompting (ablation) | Llama-3-70B-Chat · WCB Temporal Classification | performance | Hurt |
| Use few-shot prompting instead of zero-shot prompting (ablation) | Llama-3-70B-Chat · WCB Uncertainty Estimation | performance | Hurt |
| Prompt with the annotation guide instead of zero-shot prompting (ablation) | Llama-3-70B-Chat · WCB Uncertainty Estimation | performance | Hurt |
Surprising findings
- Fine-tuned PLMs consistently outperform zero-shot LLMs on stance detection, even with few-shot or annotation-guide prompting.
- FinBERT-Pretrain, despite financial pretraining, performs relatively poorly on stance detection.
Limitations
- Annotators mostly from STEM fields, possibly affecting annotation quality.
- Only 25 banks included after excluding ECB-jurisdiction banks and those with inaccessible or insufficient documents.
- Residual biases from data sources and annotation process acknowledged.
Experimental setup
Datasets: WCB corpus (380,200 sentences from 25 central banks) WCB annotated 25k sentences Czech National Bank hold-out
Benchmarks: WCB annotated dataset (Stance Detection, Temporal Classification, Uncertainty Estimation)
Metrics: weighted F1-score F1 per class
Baselines: FinBERT-Pretrain FinMA Llama-3-70B-Chat GPT-4o GPT-4.1 DeepSeek-V3 Gemini 2.0 Flash Qwen2.5-72B-Instruct-Turbo Llama-4-Scout-17B-16E-Instruct
Builds on: hawkishness measure
Closest papers by meaning
- FinBen: A Holistic Financial Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.82)
- STEER-ME: Assessing the Microeconomic Reasoning of Large Language Models (NeurIPS 2025 · similarity 0.81)
- Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows (NeurIPS 2025 · similarity 0.80)
- Towards Financial World Modeling (NeurIPS 2026 · similarity 0.80)
- Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation (NeurIPS 2025 · similarity 0.80)
- EuroSpeech: A Multilingual Speech Corpus (NeurIPS 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.