Google unveils SL2T, converts sign language to text, rolling out with Pixel 11

A way to type with your hands instead of a keyboard

Google DeepMind has unveiled SL2T, a model that converts hand movements into text in real time. The feature translating American Sign Language (ASL) into English will roll out first on Pixel 11, letting users sign sentences in front of the camera instead of typing on a keyboard within Gboard and Live Transcribe. In example screens shown by Google, signing a phrase like "movie showtimes with open captions" brings up autocomplete suggestions in the search bar.

Google said it trained SL2T by teaching the model to interpret the simultaneous movement of hands, body, and face as a whole and convert it into text. To protect privacy, the company adopted a two-part structure: hand and body pose tracking is processed entirely on-device, while the work of converting that into actual sentences is handled on the server.

Why sign-language AI has lagged until now

Sign language isn't made up of hand shapes alone. Hand position and orientation, body tilt, and facial expressions such as eyebrow or mouth shape all function simultaneously as grammatical elements. This is fundamentally different from speech-to-text translation, which processes English sentences word by word, making it difficult to simply adapt existing natural language processing models. Google explained that this technical difficulty, compounded by misconceptions about sign language, has long delayed AI development in this area.

SL2T reportedly achieved top-tier performance on academic benchmarks, and was specifically optimized for real-world use cases such as signing with one hand while holding the phone with the other. This was reportedly incorporated from early in development based on input from deaf Google employees and the AI Sign Language Advisory Committee.

The announcement coincided with Google's "Made by Google 26" event on the same day, where the company officially launched the Pixel 11. At the event, the base Pixel 11 was priced at $899, up $100 from its predecessor, while base storage doubled to 256GB. SL2T's announcement focused on accessibility features rather than hardware.

So what changes

ASL users can now search or generate conversation captions by signing with their hands instead of typing on their smartphones. Google said it plans to use ASL as a starting point to expand SL2T to other sign languages and more applications. This approach of adding convenience features while protecting privacy through on-device processing echoes Google's recently released offline speech translator, "Gemma Translator." It reflects a broader trend of accessibility technology, once focused mainly on voice recognition, expanding into hand-gesture recognition.