AI Paper Atlas

Research map / Video / Video-language understanding

From Clips to Streams: A Unified Framework for Streaming Sign Language Translation

Hoang Quan Dang, Xin Shen, Yanbin Liu, Xin Yu, Ling Chen

NeurIPS 2026 · Video › Video-language understanding · approach: Dense video captioning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Defines Streaming Sign Language Translation, which requires discovering and translating linguistic events from continuous untrimmed sign video rather than pre-segmented clips.

Problem

Sign language translation research has focused on pre-segmented clips, which disconnects models from continuous real-world signing; the paper targets translating from untrimmed streaming video.

Approach

Formally defines Streaming SLT as the joint discovery and translation of linguistic events from continuous, untrimmed video, and proposes a unified framework to address it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train translation jointly with segmentation (StreamSLST) instead of a cascaded segment-then-translate pipelineStreamSLST · BOBSLBLEU-4: 0.02 -> 0.23Improved
joint training instead of cascaded pipelineStreamSLST · Streaming-Phoenix-2014TBLEU-4: 5.86 -> 8.16Improved
joint training instead of cascaded pipelineStreamSLST · How2SignBLEU-4: 1.80 -> 2.00Improved
joint training instead of cascadeStreamSLST · Streaming-CSL-DailyMETEOR: 9.46 -> 7.34Hurt
use mBART decoder instead of deformable LSTM (ablation)StreamSLST · Streaming-CSL-Dailylocalization F1: 46.84 -> 59.37Improved
add localized visual features to query for translation (ablation)StreamSLST · Streaming-Phoenix-2014TBLEU-4: 2.89 -> 8.16Improved
let translation loss update shared queries, encoder and backbone (ablation)StreamSLST · Streaming-Phoenix-2014TBLEU-4: 5.81 -> 8.16Improved
add Tri-modal Stage-1 pretraining (ablation)StreamSLST · Streaming-Phoenix-2014TBLEU-4: 3.65 -> 8.16Improved
add Tri-modal Stage-1 pretraining (ablation)StreamSLST · Streaming-Phoenix-2014Tlocalization F1: 29.90 -> 55.95Improved
add Tri-modal Stage-1 pretraining (ablation)StreamSLST · How2SignBLEU-4: 0.95 -> 2.00Improved
use learned proposals instead of annotated oracle spans (ablation)StreamSLST · Streaming-Phoenix-2014TBLEU-4: 3.08 -> 8.16Improved
widen oracle spans by 0.5 s / 1.0 s or rescale by 1.40 / 2.25 (ablation)oracle-boundary StreamSLST · Streaming-Phoenix-2014T and How2SignBLEU-4: Phoenix 3.08 -> 3.79 at best vs 8.16 learnedNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy