New Brain Device Is First to Read Out Inner Speech

New Brain Device Is First to Read Out Inner Speech

By Emma R. Hasson edited by Allison Parshall

After a brain stem stroke left him almost entirely paralyzed in the 1990s, French journalist Jean-Dominique Bauby wrote a book about his experiences—letter by letter, blinking his left eye in response to a helper who repeatedly recited the alphabet. Today people with similar conditions often have far more communication options. Some devices, for example, track eye movements or other small muscle twitches to let users select words from a screen.

And on the cutting edge of this field, neuroscientists have more recently developed brain implants that can turn neural signals directly into whole words. These brain-computer interfaces (BCIs) largely require users to physically attempt to speak, however—and that can be a slow and tiring process. But now a new development in neural prosthetics changes that, allowing users to communicate by simply thinking what they want to say.

The new system relies on much of the same technology as the more common “attempted speech” devices. Both use sensors implanted in a part of the brain called the motor cortex, which sends motion commands to the vocal tract. The brain activation detected by these sensors is then fed into a machine-learning model to interpret which brain signals correspond to which sounds for an individual user. It then uses those data to predict which word the user is attempting to say.

On supporting science journalism

If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

But the motor cortex doesn’t only light up when we attempt to speak; it’s also involved, to a lesser extent, in imagined speech. The researchers took advantage of this to develop their “inner speech” decoding device and published the results on Thursday in Cell. The team studied three people with amyotrophic lateral sclerosis (ALS) and one with a brain stem stroke, all of whom had previously had the sensors implanted. Using this new “inner speech” system, the participants needed only to think a sentence they wanted to say and it would appear on a screen in real time. While previous inner speech decoders were limited to only a handful of words, the new device allowed participants to draw from a dictionary of 125,000 words.

“As researchers, our goal is to find a system that is comfortable [for the user] and ideally reaches a naturalistic ability,” says lead author Erin Kunz, a postdoctoral researcher who is developing neural prostheses at Stanford University. Previous research found that “physically attempting to speak was tiring and that there were inherent speed limitations with it, too,” she says. Attempted speech devices such as the one used in the study require users to inhale as if they are actually saying the words. But because of impaired breathing, many users need multiple breaths to complete a single word with that method. Attempting to speak can also produce distracting noises and facial expressions that users find undesirable. With the new technology, the study’s participants could communicate at a comfortable conversational rate of about 120 to 150 words per minute, with no more effort than it took to think of what they wanted to say.

Like most BCIs that translate brain activation into speech, the new technology only works if people are able to convert the general idea of what they want to say into a plan for how to say it. Alexander Huth, who researches BCIs at the University of California, Berkeley, and wasn’t involved in the new study, explains that in typical speech, “you start with an idea of what you want to say. That idea gets translated into a plan for how to move your [vocal] articulators. That plan gets sent to the actual muscles, and then they carry it out.” But in many cases, people with impaired speech aren’t able to complete that first step. “This technology only works in cases where the ‘idea to plan’ part is functional but the ‘plan to movement’ part is broken”—a collection of conditions called dysarthria—Huth says.

According to Kunz, the four research participants are eager about the new technology. “Largely, [there was] a lot of excitement about potentially being able to communicate fast again,” she says—adding that one participant was particularly thrilled by his newfound potential to interrupt a conversation—something he couldn’t do with the slower pace of an attempted speech device.

To ensure private thoughts remained private, the researchers implemented a code phrase: “chitty chitty bang bang.” When internally spoken by participants, this would prompt the BCI to start or stop transcribing. 

Brain-reading implants inevitably raise concerns about mental privacy. For now, Huth isn’t concerned about the technology being misused or developed recklessly, speaking to the integrity of the research groups involved in neural prosthetics research. “I think they’re doing great work; they’re led by doctors; they’re very patient-focused. A lot of what they do is really trying to solve problems for the patients,” he says, “even when those problems aren’t necessarily things that we might think of,” such as being able to interrupt a conversation or “making a voice that sounds more like them.” 

For Kunz, this research is particularly close to home. “My father actually had ALS and lost the ability to speak,” she says, adding that this is why she got into her field of research. “I kind of became his own personal speech translator toward the end of his life since I was kind of the only one that could understand him. That’s why I personally know the importance and the impact this sort of research can have.”

The contribution and willingness of the research participants are crucial in studies like this, Kunz notes. “The participants that we have are truly incredible individuals who volunteered to be in the study not necessarily to get a benefit to themselves but to help develop this technology for people with paralysis down the line. And I think that they deserve all the credit in the world for that.”