Block unveils Dessa, a voice AI trained only on speech

Block, the payments company led by Jack Dorsey, on Friday, October 23, introduced Dessa, a speech-to-speech AI model that it says was trained "from scratch on nothing but human conversation." According to the company's project page, no text or pretrained models went into it. Block calls it "the world's first AI you can have a realistic live conversation with that was trained only from human speech."theverge.com

Most voice assistants turn spoken words into text before they respond. Dessa skips that step, which puts Block's small research team on a different path from the text-anchored systems that lead the market. For now, only a small group of AI safety researchers is being offered early access.

How Dessa works

Block says Dessa is full-duplex, meaning it listens and talks at the same time. Because it learned from real conversations, it can tell when to cut in and when to backchannel with short replies such as "yeah!" or "got it!" Block also says inference runs faster than real time and that the model runs locally on a MacBook Pro M4 Max or better, according to Block. Project lead Ragavan Thurairatnam wrote on X that the model can run on consumer GPUs and called it "a GPT3 moment for speech."x.com+1

The lab describes itself as a "(very) small AI research lab at Block, founded by former members of Dessa, the Toronto deep learning startup," per the project page.block.xyz

Natural, sometimes awkward

The Verge reported that the demo conversations can be "a little awkward and uncanny." In one clip Thurairatnam shared, the model at first mistakes the speaker for someone named "Ronaldo." In another, a chat about Christopher Nolan's films drifts into Bryan Singer's Superman movies. Elsewhere, Dessa answers news of a surprise trip to Paris with what the site calls "teary gratitude." Block also published a conversation between Dessa and OpenAI's GPT-Live-1 voice model, and The Verge noted that Dessa came across as the more natural of the two.theverge.com

Block has not released latency figures, parameter counts or the size of its training data, a gap pointed out in an analysis by ic.work.ic.work

A safety argument

The team gives three reasons for building Dessa: "curiosity (aka fun), better understanding between humans and AI, and safety," The Verge reported. "Babies learn to talk without ever seeing a word written down. We wanted to see if a model could too. It can. It's just very hard," the team wrote.theverge.com

The team argues that today's voice AI "flattens what we say into text" and is "still nearly deaf to the emotion in our voices." In its words, "Imagine being in serious pain while a voice AI has no idea and just keeps going. That's the safety problem. Dessa is a step toward fixing it."

Broader public access will go through a waitlist, Unite.AI reported.unite.ai