Research map / Large language models / Language, safety and interpretability

YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpus

ICLR 2025 · Large language models › Language, safety and interpretability · approach: Sign language datasets and translation · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces YouTube-SL-25, a ~3200-hour open-domain multilingual sign language video corpus mined from YouTube with human-triaged channels, and baselines using a multilingual T5-v1.1 Small sign-to-text multitask model.

Problem

Sign language machine learning is bottlenecked by data, especially for the many low-resource sign languages beyond ASL; existing parallel datasets are small, single-language, or closed-domain.

Approach

Scale YouTube-ASL's mining pipeline across many sign languages by replacing per-video expert filtering with cheap channel-level triage by a single non-native annotator, sorted by duration, and train a unified multilingual multitask T5 model with sign language identification as an added task.

Key results

• Finetuned sgg translation reaches 37.7 BLEURT (7.5 BLEU) with full YouTube-SL-25 pretraining, versus top WMT23 score of 23.6 BLEURT (0.3 BLEU).
• ASL translation on FLEURS-ASL improves from 33.7 BLEURT (ASL subset pretraining) to 40.1 BLEURT (full pretraining), zero-shot.
• Zero-shot sign language identification on sgg rises from 54.0% (ASL pretraining) to 72.0% (full), and ssr/slf also benefit from full pretraining.

Measured effects reported in the paper

Surprising findings

• Pretraining on ASL data and finetuning on How2Sign decreased FLEURS-ASL langid accuracy relative to no sign language pretraining, with large variation across signers (18.3% vs 94.9% vs 77.0%).
• mT5 Small underperformed T5 v1.1 Small (slower convergence, worse results), so all main experiments used T5 only.

Limitations

• Triage by a non-native single annotator trades off per-video quality and long-tail channel coverage; the dataset may contain misaligned captions and some non-sign content.
• Demographic skew: dark skin underrepresented (1.9% of hours at Monk 6-10) and heavy concentration in a few countries.
• Baselines use MediaPipe pose landmarks and single-signer input, not multi-signer video; benchmarks cover only 4 sign languages.

Experimental setup

Datasets: YouTube-SL-25 How2Sign FLEURS-ASL WMT23 SignSuisse

Benchmarks: FLEURS-ASL How2Sign WMT23 SignSuisse (sgg, ssr, slf)

Metrics: BLEURT BLEU top-1 accuracy

Baselines: No sign language pretraining (T5 text-only) YT-SL-25 ASL subset pretraining WMT23 SignSuisse top system

Builds on: YouTube-ASL FLEURS-ASL baselines T5 v1.1 Whisper

Closest papers by meaning

• Scaling Sign Language Translation (NeurIPS 2024 · similarity 0.93)
• Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.90)
• LLMs are Good Sign Language Translators (CVPR 2024 · similarity 0.89)
• KD-MSLRT: Lightweight Sign Language Recognition Model Based on Mediapipe and 3D to 1D Knowledge Distillation (AAAI 2025 · similarity 0.88)
• Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues (CVPR 2025 · similarity 0.88)
• Uni-Sign: Toward Unified Sign Language Understanding at Scale (ICLR 2025 · similarity 0.88)