Researchers have developed a new brain-computer interface (BCI) that enables people with severe paralysis to communicate through both speech and gestures simultaneously using a single device. The breakthrough, led by a team at the University of California, San Francisco (UCSF), represents the first instance of a BCI controlling a full-body virtual avatar in real time.
The technology was tested on three individuals with profound paralysis, allowing them to speak and perform communicative gestures via a virtual avatar. One participant achieved perfect accuracy—100%—in decoding both speech and gestures across three separate assessments. The researchers noted that the subjects were enthusiastic about the system’s performance, with one participant even laughing upon seeing the avatar replicate his speech and gestures together.
Samantha Brosler, a researcher affiliated with UCSF and UC Berkeley, described the achievement as a significant milestone. She explained that the study demonstrates the capability to decode speech and gestural communication concurrently from a single brain implant in people with paralysis, which had not been accomplished before.
While the initial tests involved only a small number of participants, the development holds promise for individuals affected by paralysis resulting from stroke or neurodegenerative diseases such as motor neurone disease. By integrating speech and gesture decoding, the system may provide a more natural and expressive form of communication for those who have lost the ability to speak or move voluntarily.
The research team emphasized the potential impact of combining these communication modalities, which could enrich virtual interactions and improve quality of life for users. This advance in BCI technology builds on prior work focused on either speech or movement alone and marks a step toward more comprehensive neural communication interfaces. Further studies and clinical trials will be necessary to assess the system’s usability and scalability across a wider patient population.
