Sign Language Translation from Instructional Videos
Laia Tarrés, Gerard I. Gállego, Amanda Duarte, Jordi Torres, Xavier Giró-i-Nieto
2023-04-13Sign Language Translation
Abstract
The advances in automatic sign language translation (SLT) to spoken languages have been mostly benchmarked with datasets of limited size and restricted domains. Our work advances the state of the art by providing the first baseline results on How2Sign, a large and broad dataset. We train a Transformer over I3D video features, using the reduced BLEU as a reference metric for validation, instead of the widely used BLEU score. We report a result of 8.03 on the BLEU score, and publish the first open-source implementation of its kind to promote further advances.
Results
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Sign Language Translation | How2Sign | BLEU | 8.03 |
Related Papers
Speak2Sign3D: A Multi-modal Pipeline for English Speech to American Sign Language Animation2025-07-09Multilingual Gloss-free Sign Language Translation: Towards Building a Sign Language Foundation Model2025-05-30Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation2025-05-30Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation2025-05-21ADAT: Time-Series-Aware Adaptive Transformer Architecture for Sign Language Translation2025-04-16Hands-On: Segmenting Individual Signs from Continuous Sequences2025-04-11A multitask transformer to sign language translation using motion gesture primitives2025-03-25Sign Language Translation using Frame and Event Stream: Benchmark Dataset and Algorithms2025-03-09