Research map / Label-efficient and robust learning / Domain adaptation and generalization
DA-Net: A Disentangled and Adaptive Network for Multi-Source Cross-Lingual Transfer Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DA-Net, a multi-source cross-lingual transfer model with per-language disentanglers trained via a max-min feedback game and class-aware distribution-level contrastive alignment of source and target languages on mBERT.
Problem
Multi-source cross-lingual transfer suffers from language interference in a shared encoder across source languages and from language gaps between each source and the unlabeled target language, which hurt classifier performance on target languages.
Approach
Feedback-guided collaborative disentanglement trains each source's disentangler to maximize its own classifier's accuracy while pushing other branches' classifiers toward uniform predictions (weighted by MMD distances), and class-aware parallel adaptation aligns class-wise distributions per source-target pair using MMD inside a contrastive objective, with pseudo-labels from a momentum model.
Key results
- On NER (Wikiann, Setting 1 en/es/ru), DA-Net averages 72.83 vs. 69.83 for G-MOE; in Setting 2 (en/ar/zh) 75.67 vs. 71.04.
- On TEP (XNLI), DA-Net averages 63.90 vs. 62.62 for G-MOE in the Setting 1 and 2 averages reported in the table.
- On RRC (Amazon), DA-Net averages 48.61 vs. 47.29 for G-MOE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Full DA-Net (FCD + CPA) versus G-MOE on Wikiann NER | DA-Net · Wikiann NER, Setting 1 and 2 | F1 avg: +3.00 (Setting 1), +4.63 (Setting 2) vs G-MOE per text | Helped |
| Full DA-Net versus G-MOE on XNLI and Amazon | DA-Net · XNLI TEP and Amazon RRC | accuracy avg: +1.28 (XNLI Setting 1), +1.32 (Amazon) vs G-MOE per table | Helped |
| Removing FCD (disentanglers) from DA-Net (ablation) | DA-Net · Wikiann NER, en target (af) | F1: af decreases by 2.16% without FCD | Helped |
| Removing CPA from DA-Net (ablation) | DA-Net · Wikiann NER, target id | F1: id decreases by 6.25% without CPA | Helped |
| Replacing distribution alignment with supervised contrastive learning in CPA (ablation) | DA-Net · Wikiann NER and RRC | F1 / accuracy: slight drop | Helped |
| Using MMD between language distributions instead of class-aware MMD in CPA (ablation) | DA-Net · Wikiann NER and RRC | F1 / accuracy: slight drop | Helped |
| Adding FCD disentanglers to the shared encoder (per-source disentanglement) (ablation) | DA-Net, Wikiann NER · Source-branch F1 for en, ar, zh | F1 | Helped |
| Class-aware parallel adaptation (CPA) on Amazon RRC target languages in Setting 2 | DA-Net · Wikiann NER, fa and ur targets (Setting 2) | F1: +18.64 (fa) and +17.56 (ur) vs SOTA per text | Helped |
| Adding DA-Net on Amazon RRC with a different source set | DA-Net · Amazon RRC, Setting 1 | accuracy avg: +1.62 vs mBERT | Helped |
Surprising findings
- The paper reports that a prior method's uniform alignment of all languages can eliminate shared pair-specific features, which motivates its parallel per-pair adaptation.
Limitations
- XNLI training uses only 20,000 samples per class per language due to computational limits.
- Gains over G-MOE are modest on several settings (e.g., RRC Setting 2 de is slightly lower than MulTS).
Experimental setup
Datasets: Wikiann Amazon Reviews Corpus XNLI
Benchmarks: Wikiann Amazon Reviews Corpus XNLI
Metrics: token-level F1 accuracy
Baselines: mBERT M-MOE MAN MulTS G-MOE
Builds on: mBERT MAN MulTS G-MOE M-MOE
Closest papers by meaning
- Cross-Lingual Transfer with Class-Weighted Language-Invariant Representations (ICLR 2022 · similarity 0.87)
- ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality (ICLR 2026 · similarity 0.86)
- Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models (ICLR 2025 · similarity 0.85)
- Constrained Decoding for Cross-lingual Label Projection (ICLR 2024 · similarity 0.84)
- When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners (NeurIPS 2025 · similarity 0.84)
- XCOT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.