Google launches sign language-to-text AI on Pixel 11

Google Alphabet Inc. ↗ 0.46% DeepMind today introduced a sign-language-to-text model called SL2T, marking the first time sign language AI has shipped inside a consumer smartphone feature. The model powers sign-to-text dictation in Gboard and Live Transcribe on the Pixel 11, launching with American Sign Language to English translation.deepmind+1

The feature allows Deaf and hard of hearing users to sign into their phone's camera anywhere they would normally type — searching the web, drafting messages, querying Gemini, or responding in face-to-face conversations through Live Transcribe. Google says testers found signing in ASL faster and more natural than typing in English.siliconangle+1

How It Works

SL2T was trained on more than 100,000 hours of signing data spanning over 50 sign languages, with roughly a quarter in ASL. Rather than processing raw video, an on-device model called MediaPipe Holistic tracks 130 key points on the signer's face, body, and hands, sending only geometric coordinates to Google's servers. The original camera feed is discarded immediately.unite+1

The model translates directly from pose coordinates to text, bypassing intermediate annotations known as glosses that prior systems relied on. "Glosses fail to capture rich, non-linear aspects of sign languages such as non-manual markers and spatial constructions," DeepMind said. "Translating directly from landmarks removes artificial vocabulary limits and allows translation quality to scale directly with data."engadget

On the FLEURS-ASL benchmark, SL2T achieves a zero-shot score of 70 BLEURT, which Google describes as well above any previously reported result.deepmind+1

Governance and Limitations

Google established the AI Sign Language Advisory Committee, bringing together organizations including the National Association of the Deaf and the World Federation of the Deaf. A co-authored impact report defines SL2T 1.0 as an assistive tool for low-stakes settings and explicitly rules out use in medical, legal, police, or employment contexts. The report states the tool does not satisfy legal obligations for reasonable accommodations under the Americans with Disabilities Act.unite

Known limitations include degraded accuracy in low light, no support for multiple signers, a 60-second input cap per clip, and no conversation memory across clips. The model was not trained on signers under 18.unite

The project originated with Sam Sepah, a Deaf Googler, and Deaf perspectives shaped every stage from data collection to deployment. Google plans to expand to additional sign languages and devices, with sign language generation listed as active research. The feature is available at no additional cost.siliconangle+1