Research map / Large language models / Language, safety and interpretability
FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fuse translated source-target language representations in intermediate layers instead of translate-train alone | XLM-R on XTREME · XTREME | Avg: 68.2 -> 77.0 | Helped |
| Add intermediate cross-language fusion rather than translating and training on separate single-language examples | XLM-R on PAWS-X · PAWS-X | accuracy: 90.4 -> 91.2 | Helped |
| Add intermediate cross-language fusion rather than translate-train | XLM-R on MLQA · MLQA | F1 / EM: 72.8 / 54.3 -> 75.8 / 57.2 | Helped |
| Add self-teaching loss to FILTER (ablation) | FILTER on POS · POS | F1: 75.5 -> 76.2 | Helped |
| Add self-teaching loss to FILTER (ablation) | FILTER on NER · NER | F1: 66.7 -> 67.7 | Helped |
| Add self-teaching loss to FILTER (ablation) | FILTER on XNLI · XNLI | accuracy: 83.6 -> 83.9 | Helped |
| Add self-teaching loss to FILTER (ablation) | FILTER on PAWS-X · PAWS-X | accuracy: 91.2 -> 91.4 | Helped |
| Add self-teaching loss to FILTER (ablation) | FILTER on XQuAD · XQuAD | F1: 82.3 -> 82.4 | Helped |
| Add self-teaching loss to FILTER (ablation) | FILTER on MLQA · MLQA | F1: 75.8 -> 76.2 | Helped |
| Add self-teaching loss to FILTER (ablation) | FILTER on TyDiQA-GoldP · TyDiQA-GoldP | F1: 68.1 -> 68.3 | Helped |
| Replace intermediate fusion with simple concatenation from the input layer (ablation) | FILTER on PAWS-X · PAWS-X | accuracy: 2.5 points | Hurt |
| Replace intermediate fusion with simple concatenation from the input layer (ablation) | FILTER on POS · POS | performance: 15.2 points | Hurt |
Closest papers by meaning
- Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework (ICLR 2020 · similarity 0.90)
- XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation (ICML 2020 · similarity 0.89)
- Constrained Decoding for Cross-lingual Label Projection (ICLR 2024 · similarity 0.88)
- CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer (AAAI 2024 · similarity 0.88)
- Enhancing Non-English Capabilities of English-Centric Large Language Models Through Deep Supervision Fine-Tuning (AAAI 2025 · similarity 0.88)
- Soft Token Alignment for Cross-Lingual Reasoning (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.