Research map / Video / Video generation
Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Sign-IDD, a diffusion framework for gloss-to-pose sign language production that converts 3D joints to 4D bone representations (direction + length) and uses attribute separation/control layers to guide generation.
Problem
Gloss-to-pose (G2P) generation for sign language production treats poses as absolute 3D joint coordinates, neglecting relative positions among joints, which causes finger detail confusion and limb misorientation.
Approach
Disentangle each 3D joint into a 4D bone representation (unit direction vector plus Euclidean length between parent and child joints), feed it alongside 3D coordinates into a gloss-conditioned diffusion denoiser with attribute separation and control layers, and add a bone-orientation loss.
Key results
- On PHOENIX14T TEST, Sign-IDD reaches BLEU-1 24.80 and BLEU-4 9.08, versus 22.03 BLEU-1 for GCDM.
- Ablation on PHOENIX14T TEST: Base WER 84.74 drops to 76.66 with both ID and ACD modules.
- On USTC-CSL, Sign-IDD achieves MPJPE 72.20 versus 79.27 for D3DP-sign and BLEU-1 65.26 versus 59.37.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ID to the diffusion baseline (ablation) | Sign-IDD Base · PHOENIX14T TEST | BLEU-1: 21.50 -> 23.46 | Helped |
| Add ID to the diffusion baseline (ablation) | Sign-IDD Base · PHOENIX14T TEST | WER: 84.74 -> 77.53 | Helped |
| Add ACD to the diffusion baseline (ablation) | Sign-IDD Base · PHOENIX14T TEST | BLEU-1: 21.50 -> 23.42 | Helped |
| Add ACD to the diffusion baseline (ablation) | Sign-IDD Base · PHOENIX14T TEST | WER: 84.74 -> 79.92 | Helped |
| Add both ID and ACD to the diffusion baseline (ablation) | Sign-IDD Base · PHOENIX14T TEST | BLEU-1: 21.50 -> 24.80 | Helped |
| Add both ID and ACD to the diffusion baseline (ablation) | Sign-IDD Base · PHOENIX14T TEST | WER: 84.74 -> 76.66 | Helped |
| Set bone-loss weight λ to 0.01 instead of 0.1 (ablation) | Sign-IDD · PHOENIX14T DEV | BLEU-1: 25.40 -> 21.34 | Hurt |
| Set bone-loss weight λ to 1 instead of 0.1 (ablation) | Sign-IDD · PHOENIX14T DEV | BLEU-1: 25.40 -> 20.85 | Hurt |
| Set diffusion timestep t to 500 instead of 1000 (ablation) | Sign-IDD · PHOENIX14T DEV | BLEU-1: 25.40 -> 24.11 | Hurt |
| Set diffusion timestep t to 1500 instead of 1000 (ablation) | Sign-IDD · PHOENIX14T DEV | BLEU-1: 25.40 -> 23.09 | Hurt |
| Set inference sampling steps i to 1 instead of 5 (ablation) | Sign-IDD · PHOENIX14T DEV | BLEU-1: 25.40 -> 24.05 | Hurt |
| Set inference sampling steps i to 10 instead of 5 (ablation) | Sign-IDD · PHOENIX14T DEV | BLEU-1: 25.40 -> 20.85 | Hurt |
Limitations
- Ablations are reported only on PHOENIX14T; USTC-CSL is not used for ablation.
- Target poses for PHOENIX14T are pseudo-labels from OpenPose 2D keypoints lifted to 3D, so ground truth is approximate.
- Evaluation relies on a back-translation SLT model (NSLT), whose accuracy bounds the semantic metrics.
Experimental setup
Datasets: PHOENIX14T USTC-CSL
Benchmarks: PHOENIX14T USTC-CSL
Metrics: BLEU-1 BLEU-4 ROUGE WER FID MPJPE MPJAE
Baselines: PT-base PT-GN NAT-AT NAT-EA DET GEN-OBT D3DP-sign G2P-DDM GCDM
Builds on: denoising diffusion probabilistic models G2P-DDM GCDM D3DP
Closest papers by meaning
- Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization (NeurIPS 2025 · similarity 0.87)
- VSNet: Focusing on the Linguistic Characteristics of Sign Language (CVPR 2025 · similarity 0.87)
- G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Model (AAAI 2024 · similarity 0.87)
- Neural Sign Actors: A Diffusion Model for 3D Sign Language Production from Text (CVPR 2024 · similarity 0.87)
- Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment (NeurIPS 2026 · similarity 0.86)
- Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.