Never would have thought that a major breakthrough in brain-computer interface would come so soon!
It even made the cover of Nature.
A paraplegic patient is using his “mind” to type out a paragraph, and can output a letter in about 0.5 seconds.
The accuracy rate is also very impressive, up to 99.1%.
All he needs to do is to “handwrite” the letters in the brain, and then the system will automatically identify the letters generated, a minute to write 90 characters.
Although the writing is not very good, but at least by Nature’s “favor”.
It is worth mentioning that before this, the patient in another test project, tried to “think” to move the cursor to type, but a minute can only type 13.4 correct characters.
The study has generated a lot of interest from the academic community and the Internet.
One professor at the University of Washington even exclaimed, “I can type slower than that!
The RNN took credit for the implantation in the brain of the old man codenamed T5, which is an array of two electrodes from Braingate, each containing 96 electrodes.
The first difficulty was encountered right at the beginning of the experiment: how to recognize when the user starts trying to write letters.
It was eventually discovered that a model originally used for speech recognition could accomplish this task.
After solving this problem, the researchers found that the brain activity observed when writing a single character was relatively constant and always focused on it.
And the areas used to write similarly shaped letters such as “b” and “p” were close together.
It seems that the neural representation of handwriting in the motor cortex does not fade even after years of paralysis.
After manual annotation, these data were ready to be used as the original data set.
The next step was the algorithm, and the researchers chose Recurrent Neural Network (hereinafter referred to as RNN).
Compared to common feedforward neural networks, RNNs perform the same task for each element in a data sequence, and the computation results depend on all previous results, hence the recurrent name.
RNN is better at predicting continuous data, which is just right for this study of writing a sentence continuously.
Although RNN is powerful but has a disadvantage that it needs a large amount of data, otherwise it is prone to overfitting.
The only subject involved in this study was the old man, and he was not willing to spend several hours a day doing a lot of repetitive writing to provide data ????.
But it does not matter, there is data augmentation (Data Augmentation). It is to make some small changes to each image data, rotate a little, zoom a little, or mirror flip and other operations to increase the diversity of data.
In addition to the 26 letters, input English also has to have some necessary punctuation. For example, spaces, the researchers asked the old man to use > instead, the English period only a point is not easy to distinguish, with ~ instead. There are also commas, periods and question marks.
But this study did not include numbers, probably because the researchers found it a little difficult to distinguish between z and 2, so they left it for another time to solve.
The data used at the beginning of the training was only 242 sentences, and then a few more were added each day, ending up with a total of 572 sentences and 31,472 characters.
Finally, to address the fact that some of the English letters are too similar to each other, the researchers also designed a set of alphabet for testing specifically for brain-computer interface, which will have a much higher accuracy rate, but with learning costs.
Character accuracy rate of up to 99.1% then came the volunteer testing stage.
According to the screen prompts, volunteers in the brain to copy a letter by letter writing, characters after recognition generated on the screen.
The results of the test showed that there was a delay of about 0.4-0.7 seconds between the time the brain “hand-wrote” the characters and the time they appeared on the screen.
Overall, the volunteers could type an average of 18 words and 90 characters per minute, with a character error rate of only 5.9%.
After a predictive language model similar to the phone’s auto-correction, they further increased the accuracy rate of characters to 99.1%.
The error rate for words was also reduced from 25.1% to 3.4%.
In addition, the volunteers also did some self-creation – without copying, they “wrote” their own sentences and were able to type 73.8 characters per minute, with an accuracy rate of over 97% after correction by the predictive language model.
Finally, to push the limits, the researchers also trained a new RNN, where the user writes the entire sentence and then centralizes the process, resulting in a 99.83% accuracy rate, although the user does not get real-time feedback.
In fact, this is actually part of the BrainGate project. It’s a multi-institutional consortium that includes Brown University and the U.S. biotech company Cyberkinetics, focusing on brain-computer interface technology that works to restore communication, mobility and independence to people with neurological disorders, injuries or loss of limbs.
Previously, this project enabled wireless transmission of brain-computer interface signals, allowing patients to leave the laboratory environment and easily watch videos online at home.
Krishna Shenoy, a researcher at the Howard Hughes Medical Institute (HHMI) at Stanford University and one of the paper’s authors, said the biggest innovation of the study was deciphering the brain signals associated with handwritten notes, allowing paraplegics to type quickly and accurately.
Paper one author, Dr. Frank Willett, also from Stanford University, said he would open source the entire study’s code and neural data.
For now, it’s not a complete, clinical commercial system, and after all, it has only been tested on one person.
Next will be enhancements in more test groups, expansion of typing functions (editing, deletion), and expansion of character sets (e.g., capital letters, and other languages).
Is it possible to DREAM a Chinese language?
Beyond that, there are some factors worth discussing, such as cost and risk.
Pavithra Rajeswaran, a scholar in the Department of Bioengineering at the University of Washington, and Amy Orsborn, a scholar in the Department of Electrical and Computer Engineering, said the study still needs to be tested to demonstrate whether the costs and risks of implanting electrodes into the brain are justified.
The paralyzed are not the only ones who will benefit. In addition to the paralyzed, there are also people who have difficulty typing due to other injuries and illnesses who say they are thrilled!
An example is atresia syndrome, where damage to part of the nerves leads to the degeneration or loss of some body functions, making it impossible to communicate through speech although conscious.
There is also repetitive stress injury (RSI), which includes mouse hand and tendonitis resulting from improper mouse use or typing.
One RSI patient said, I also need a function that can mimic the mouse wheel, but the netizens replied to him that you use a foot pedal or eye tracking to be much simpler than an invasive brain-computer interface.
Seeing this message, netizens also had a brainstorming session.
Some people found that imagine yourself writing with your hands and directly imagine the trajectory in your head is not the same feeling, I wonder which is easier to identify.
How about you also try to write in the brain?
Recent Comments