BREAKTHROUGH “speech neuroprosthesis”, developed by researchers at UC San Francisco (UCSF), California, USA, has allowed a man with severe paralysis to communicate using sentences. By translating signals from the brain intended for the vocal cord the neuroprosthesis can articulate words as text on a screen.
“To our knowledge this is the first successful demonstration of direct decoding of full words from the brain activity of someone who is paralysed and can’t speak” said Edward Chang, the Joan and Sanford Weill Chair of Neurological Surgery at UCSF, Jeanne Robertson Distinguished Professor, and senior author of the study.
Chang’s study offers a new approach to neuroprosthetics, translating signals intended to control the muscles of the vocal system, instead of spelling-based approaches that type out letters one-by-one in text. Chang has said that this approach taps into the natural and fluid aspects of speech allowing for more rapid communication of information than the spelling-based approaches, which are considerably slower and more laborious.
The technology was developed by mapping the cortical activity patterns associated with the vocal tract movements that produce each consonant and vowel. New methods of real-time decoding of these patterns were used to translate findings to recognition of full words. In order to test his technology in patients with paralysis, Chang launched the BRAVO study.
BRAVO1, the first participant, was a man who suffered a brainstem stoke over 15 years ago, causing extremely limited head, neck. and limb movements since the injury. Working with researchers, BRAVO1 created a 50-word vocabulary that Chang’s team could recognise from his brain activity. A high-density electrode was surgically implanted into BRAVO1’s speech motor cortex, after recovery, and over several months, researchers recorded over 48 sessions of neural activity whilst BRAVO1 would attempt to say the 50-words.
To translate the recorded neural patterns researchers used custom models of neural networks combined with forms of artificial intelligence. The approach was tested by presenting sentences from the 50 words and asking BRAVO1 questions. The team found that their system was able to translate words at a rate of up to 18 words a minute, with 75% median accuracy from brain activity.
Chang and his team have emphasised the importance of their continued work and looking forward will expand their trial to include more participants with a widened vocabulary.
