이미지: X — 프론티어랩
Summary
- Google DeepMind has unveiled SL2T, a model that converts sign language into text
- American Sign Language (ASL)-to-English translation will first appear in Pixel 11's Gboard and Live Transcribe
- Hand-motion tracking runs on-device, while text conversion is processed on the server
- 모델명
- SL2T (Sign Language-to-Text)
- 개발사
- Google DeepMind
- 1차 지원 언어
- 미국식 수어(ASL) → 영어
- 적용 기기·기능
- Pixel 11, Gboard, Live Transcribe
- 처리 방식
- 포즈 추적은 온디바이스, 텍스트 변환은 서버
- 개발 협력
- 청각장애인 구글 직원 및 AI Sign Language Advisory Committee
A way to type with your hands instead of a keyboard
Google DeepMind has unveiled SL2T, a model that converts hand movements into text in real time. The feature translating American Sign Language (ASL) into English will roll out first on Pixel 11, letting users sign sentences in front of the camera instead of typing on a keyboard within Gboard and Live Transcribe. In example screens shown by Google, signing a phrase like "movie showtimes with open captions" brings up autocomplete suggestions in the search bar.
Google said it trained SL2T by teaching the model to interpret the simultaneous movement of hands, body, and face as a whole and convert it into text. To protect privacy, the company adopted a two-part structure: hand and body pose tracking is processed entirely on-device, while the work of converting that into actual sentences is handled on the server.
Why sign-language AI has lagged until now
Sign language isn't made up of hand shapes alone. Hand position and orientation, body tilt, and facial expressions such as eyebrow or mouth shape all function simultaneously as grammatical elements. This is fundamentally different from speech-to-text translation, which processes English sentences word by word, making it difficult to simply adapt existing natural language processing models. Google explained that this technical difficulty, compounded by misconceptions about sign language, has long delayed AI development in this area.
SL2T reportedly achieved top-tier performance on academic benchmarks, and was specifically optimized for real-world use cases such as signing with one hand while holding the phone with the other. This was reportedly incorporated from early in development based on input from deaf Google employees and the AI Sign Language Advisory Committee.
The announcement coincided with Google's "Made by Google 26" event on the same day, where the company officially launched the Pixel 11. At the event, the base Pixel 11 was priced at $899, up $100 from its predecessor, while base storage doubled to 256GB. SL2T's announcement focused on accessibility features rather than hardware.
So what changes
ASL users can now search or generate conversation captions by signing with their hands instead of typing on their smartphones. Google said it plans to use ASL as a starting point to expand SL2T to other sign languages and more applications. This approach of adding convenience features while protecting privacy through on-device processing echoes Google's recently released offline speech translator, "Gemma Translator." It reflects a broader trend of accessibility technology, once focused mainly on voice recognition, expanding into hand-gesture recognition.
