Scientists announced Wednesday they were able to help a person with paralysis translate their imagined handwriting into text through a brain–computer interface (BCI) that was faster than other types of assistive communication.
Why it matters: While the interface was only tested in one person and is a proof-of-concept finding, some experts say it's an "important milestone" in developing the technologies needed by millions of people globally who've lost the ability to use their upper limbs or the ability to speak due to paralysis, strokes, or Lou Gehrig's disease.
Details: The team of researchers, whose trial is part of an international collaboration called BrainGate2, implanted electrodes on the surface of the brain of the participant (who is paralyzed from the neck down) to study the complex patterns of neural activity used when visualizing the task of handwriting individual letters.
- They were able to decode the electrical activity from about 200 different neurons into a prediction of what letter the person was wanting to make, says study co-author Krishna Shenoy, a professor of engineering at Stanford University.
- "Basically what that means is that when you're making the shape of a letter, you get a very unique pattern of electrical activity that [co-author Frank Willett's] algorithms that are based in machine learning can readily interpret," Shenoy says.
What they found: The participant was able to compose sentences and communicate at a rate of about 90 characters per minute with a 94% raw accuracy and 99% accuracy with autocorrect, according to the results published in Nature.
- This compares with existing communication BCIs that use the brain to "point-and click" on letters at a pace of about 40 characters per minute.
- "It's cool to finally be able to get speeds that are comparable to normal handwriting or comparable to smartphone typing in this age group," Frank Willett, who is a research scientist at Stanford, tells Axios.
What they're saying: Jennifer Collinger, associate professor at the University of Pittsburgh's Rehab Neural Engineering Lab who was not part of this study, says the findings are "exciting and interesting" for several reasons.
- One is that from a practical communications standpoint, the new technique appears to double the current rate of assistive communications, Collinger says.
- "But, more than that, I would not have thought to try to decode handwriting. It seems like a very challenging problem: How is that going to be better than accessing an onscreen keyboard that people have been working towards for decades? The fact they were able to achieve such a high level of performance is really, really interesting," she says.
- "They were able to show if you use both direction information and temporal variability you can get very responsive, very accurate performance," Collinger adds.
What we're watching: "The future really is — as we learn more and more about the brain — [that] we should be able to interact with it and help overcome dysfunction as well as to understand how it normally functions," Shenoy says.
- This is an "important milestone in the development of BCIs and machine-learning technologies that are unraveling how the human brain controls processes as complex as communication," NIH BRAIN Initiative director John Ngai said in a statement. NIH helped fund the study.
Go deeper: Watch a video on the BCI handwriting experiment from Nature Press on YouTube.