Brain implant lets man with complete paralysis spell out thoughts
science.orgI started (and quit) a PhD based on the Machine Learning algorithms used for BCI systems such as this.
Every 6-12 months, someone will write a puff piece about their research in order to get grant money[1]/street cred. And every time, HN responds as if it's sci-fi tech from the future and I shit on everybody's dreams.
So here we go:
At a glance, nothing about this system in particular is new. It seems to solve the same problems as older invasive arrays from 15 years ago, with the same deal-breaking flaw still unfixed (namely, the body rejecting the implant over time).
He had OK-ish performance in a single patient (one character per minute on a good day), but single-patient performance is misleading as it's very common for a system to work with one "golden" patient but not others. Interestingly, no real research has been done (at least, none as of 4 years ago) to look into why this is the case - most "breakthroughs" are people throwing machine-learning spaghetti at the wall to cover up deliberately flawed benchmarking. (To be fair, this is more of a comment on the field as a whole than this particular trial. His results seem honest and humble.)
The field is going nowhere, and this article is yet another shining example of this. Nothing will ever be achieved until we abandon the current paradigm and actually learn more about the underlying neurophysiology (or possibly create batshit-insane sensors).
[1] They actually explicitly mention this in the article this time, which is pretty bold:
>Chaudhary’s foundation is seeking funding to give similar implants to several more people with ALS. He estimates the system would cost close to $500,000 over the first 2 years.
As pointed out by hprotagonist in a comment below, there have also been claims of research misconduct claims made against the principle researchers involved with this group. So not exactly promising. https://news.ycombinator.com/item?id=30773013
Would you kindly care to explain how the field should change paradigm? Do you mean that " black boxing" the brain with input/output and hoping something will emerge is a non-productive endeavour, and that the neurophysiology, like a bottom up approach from biology/neuroscience, is the way to go?
I'm a comp-sci student too, and considering what possibilities there are for studying BCI, machine intelligence and the brain. But I think that maybe, having approached it from a more biologically oriented angle would have been better. After having read "On intelligence" I'm just more aware how disconnected our modelling of intelligence and the brain/BCI is. On one hand, there's just "throw more computing power at it", and the other, there's cutting a fruit fly's brain into slices and building a special machine just to do so, while afterwards scanning the resulting pathways [0]. We can't even understand brains way less complex than our own, so I fear that a bottom up approach is way beyond my lifetime. I remember a Danish neuroscientist once said, that there's a higher chance to understand every star in the universe, before we understand the inner workings of the human brain.
Do you mind me asking, what did you do instead?
[0] https://ai.googleblog.com/2019/08/an-interactive-automated-3...
>Do you mean that " black boxing" the brain with input/output and hoping something will emerge is a non-productive endeavour, and that the neurophysiology, like a bottom up approach from biology/neuroscience, is the way to go?
I think what we should be doing is something different to what we are now. The current "throw machine learning at things and expect it to magically work" paradigm has achieved nothing more than bullshit non-results for two decades straight, and unfortunately this is where a disproportionate amount of the grant money/researcher's eyeballs go.
BCI research, at least on the algorithms side, is basically a "bullshit job" where you do something you know will fail, it fails, and then you scream from the rooftops how successful you were until grant money gets thrown at you.
>Do you mind me asking, what did you do instead?
I got a real job as an SWE. It was at a really good company (Blackmagic Design in Port Melbourne) with a fantastic, highly politically incorrect culture. Quit that job just before the pandemic to run my own business and be a dad, but still get nostalgic about it to this day.
Thanks for the honest answer, which appealed to me because I’ve felt that way as a spectator for the last couple of decades.
Thanks for the answer! I've always been fascinated with intelligence and the brain, but most stuff in machine learning / AI just seems so disconnected from actually trying to understand intelligence or the brain on any level, and has become it's own thing more akin to pure optimization but with less understanding of the inner process mathematically, where the word AI and related are just thrown around without any meaning, in order to suck up interest and funds. Maybe it is scary for people to admit, the this whole thing built around "AI" as it is referred to in popular terms today, is not the only way to go, and does not have infinite potential. Maybe be more humble in what it can do, and simultaneously be open to other paths of research, even if it means we are starting with not knowing that much at all. But being moderate might not be the best way to ensure capital, unfortunately.
I never forget my stint in a research group somewhat related to AI, where the professor spoke in front of an audience of politicians to get funding, and just threw around bullshit terms and mixed in, what is in reality, words describing very technical subjects which require some context to actually understand. Every one of the politicians just sat there, nodding, like they understood. While we watched from telly in our office. I think part of my faith in our government died that day, together with a healthy dose of respect for public research. At least it taught me to be way more picky in what science represents in our media and society. It's interesting how on some level, popularity contests never really go away... I really yearn for some kind of structure, where income for research is not based on popularity and internal/external power struggles, but somehow more diverse and fairly shared. TBH it has a lot to do with general education and interest, while at the same time, somehow overturning the current structure where in reality, research groups face fairly hard and unforgiving adversarial conditions, instead of more cooperative ones. Oh well, resources are not infinite, but how we choose to use them can be debated.
Sounds like an awesome place to work! I'm sure if you left on good terms, you could always return. But running your own business is usually a passion project, so I hope it works out and it won't be necessary! Also being a dad, it's easier to prioritize your children, once you can manage your schedule more freely. I'm not a parent myself, but jumped the gap on 30 this year heh, and from what I know from my earlier colleagues who have children, and are mindful of how they spend their time, you become way more averse to bullshit with the little time you have, for your own stuff and how you provide for your family. I've always loved to do research and development, it doesn't really matter what it is, so I'm considering joining a normal SWE company too after UNI, and see where the future takes me, a bit akin to what you are doing.
When I was in neuroscience I came to the same conclusion. People just keep trying new ways to do information processing which is really easy and fun. Solving the biology and material science problems are fucking hard and high risk, as in, if you try you’ll probably fail. It’s going to be a very long time before these things are solved. Not to criticize because I’m also too cowardly to try tackling those problems for the aforementioned reasons.
I wonder if some kind of wearable fMRI could be used to just avoid the whole thing. Seems like the universal rule of tech is that any insane idea within the laws of physics is eventually possible, but anything to do with biology is 50x harder
If we discover room temperature superconductors this might be in the cards. Lots of exotic stuff becomes possible then but no one knows if it’s possible to have room temperature superconductors.
this is fascinating, and I thank you for your time in it, even if you dipped out of it in the end.
Question, if you've got the bandwidth - I've got a cochlear implant which feels like it's at least a little tangential in terms of hooking into the electrodes and auditory nerves ... been wearing this wonderful thing for 30 years, and I'm curious as to why the body would reject his implant versus mine (and other folks who wear CI's)?
I couldn't tell you why the CI is accepted long-term, but the penetrating electrode arrays like the one in the article actually puncture the surface of the brain with tiny needles and cause the surrounding tissue to scar/calcify.
There are non-penetrating arrays that sit just above the brain (avoiding this problem to a large degree), but they don't have anywhere near the precision of the penetrating sensors.
Will Neuralink have the same problem? Seems like that would put a big damper on the insane level of hype around it.
Curious why they go through each letter saying yes or no, instead of just teaching them how to pick letters based on a binary output (ie: a= 00000, b=00001)? you'd think the characters per minute would increase dramatically.
Morse code, where dot is ‘think about tennis’ and dash is ‘think about home’?
https://amp.theguardian.com/science/2010/feb/03/vegetative-s...
It occurs to me that this is a binary search on the alphabet, but could be made into a much more human-compatible format: “Is the letter in the first half of the alphabet?” “Is the letter between A and F?”
I'm not sure that's exactly human friendly. Can people answer questions like that without training? Would the amount of thought needed create a noise signal?
If they have vision, you could ask "Which of these two sets is the letter in", where the sets were dynamically generated by predictive software.
yeah, I guess you'd start with the first 13/last 13 and break it down in halves from there. It would take only 6 choices to pick a letter. I still think memorizing the binary version of the letters would be faster, and wouldn't take all that long considering the amount of use.
The code is (or can be) the same either way. I imagine after a few days the user would memorize the codes for the letters and skip the prompts.
On a related note, one character per minute is about the same speed as the first trans-atlantic telegraph cable in 1858.
You want a variable length code that takes into account letter frequencies.
There’s relevant scenes to this in the Stephen Hawking biopic, The Theory of Everything.
I feel like this is science research in general - it's a battle for money, and only a few groups really manage great research vs. doing the whole grant application shuffle.
What do you think about the few groups actually doing electrode design (i.e. there was a UC Berkeley/Lawrence Livermore group looking at electrode designs)?
Vs. I think most of the innovations in BCI engineering happen to be in the commercial sector (i.e. deep brain stimulation electrodes from Boston Sci, Medtronic, Abbott, the arrays from Black Rock in Utah, Neuralynx, etc).
Any thoughts on Neuralink's work/approach?
The promise is significantly reduced device costs for far superior electrode counts. Even if it can't interpret yet, the ability to get a more nuanced look at the firing of neurons should be a huge benefit to help increase understanding
Haven't really looked into it, but the big demo where the pig would walk on the treadmill and the model tracked its movement really set off my bullshit radar (high signal strength of muscle actuation, cyclical nature of the signal - it's not at all what you'd show to convince someone who was familiar with the literature that it's not just more bullshit snake oil).
The monkey playing the computer game was the more interesting one imo - but it’s outside of my field of expertise.
I wish all that grant money was spent on trying to cure the underlying diseases.
It's conceivable research like this might provide insights into the nature of these conditions that does help lead us to treatments. Anyway if you think a cure might be decades away but work like this can be done affordably now, it just makes sense.
I don't really know much about this field. Can you say more about what "batshit-insane sensors" might entail?
https://news.ycombinator.com/item?id=30773013 Previously:
The study comes in the context of past findings of serious misconduct against Birbaumer and Chaudhary. The findings concerned the data and analysis in two previous papers published in PLoS Biology. The two articles, subsequently retracted, also concerned the use of brain activity to decode the thoughts of completely locked-in patients. The German research agency, Deutsche Forschungsgemeinschaft (DFG) found that the scientists failed to show complete analysis of their data and patient examinations in these previous studies and made false statements. The allegations do not relate to the findings of the current research which involved different methodology, supervision and analysis. In a statement to Technology Networks, Birbaumer said that the new study “shows that all accusations are wrong” and suggested that additional forthcoming legal developments would further exonerate his and Chaudhary’s prior research.
Has anyone else noticed this is sort of a permanent story? I’ve read it as news verbatim at least once a year since the 90s. It’s the “Russian bombers intercepted” of medical science. This article is better than most, acknowledging prior work but only to a few years.
Not sure if relevant but it's good to distinguish techniques that are in the end based on muscle activity, which, even if the tiniest ones around the eye, give VERY strong electrical signals compared to brain activity. That's what basically makes the 'consumer variants' tick as well.
This article seems to be actually, entirely, and directly based on brain activity.
Are you sure it’s the same story?
Or are we seeing reports of incremental improvements to technologies for communicating with locked-in people.
There is a difference between using a brain implant and other devices like eye trackers and some type of muscle translator.
I've seen a lot of the latter. I don't think I've ever seen the former.
"I would like to listen to the album by Tool loud"
That sentence speaks volumes to me.
> he made dozens of sentences at a painstaking rate of about one character per minute
It's a good thing he wanted a band with a nice short name. Imagine if he was a fan of these guys https://www.metal-archives.com/bands/Paracoccidioidomicosisp...
https://www.metal-archives.com/bands/Eximperituserqethhzebib... has entered the chat.
I hope that somebody got the man a stereo, and turned it way up. Not just some random bluetooth speaker crap, like a real stereo. Loud, full and clear.
We don't know which Tool album he wanted, although various Rock/Metal sites that posted this article wondered if perhaps the lyrics to "Parabola" might be resonating with him at the moment.
Might as well play them all, it's not like he has anything better to do.
Honestly seems like the perfect choice considering Maynard Keenan's mother, and her struggles being the subject of a bunch of his songs.
10,000 Days couldn’t be more fitting of a request. But given how long he’s been out I think the humane choice would be Fear Inoculum so he could finally get caught up!
Would be cool if some members of Tool visited him when they're in the country.
Just don't tell him about Fear Inoculum. It's better that way.
Unbelievably hopeful and tragic at the same time. That he could learn to control electrode implants to communicate (in an albeit "rudimentary" way) is just incredible, but I can't imagine the feelings of unfairness at such a young age for him. Hopefully there's a path here to help others communicate much longer than they'd otherwise be able to.
One of the other sentences he made out was "I love my cool son," which hit particularly close to home given I'm the same age as well. Oy.
One more step towards Lock In[0] sort of becoming real.
Isn’t that what Musk’s Neuralink is doing? They automated the invasive surgery to install the device, and their device has thousands of wires to read neuronal signals.
Is there any peer-reviewed literature on what technology or advances Neuralink has? It's purported to do these things, but a lot of what I've seen is skeptical.
There are a few publications by people with some connection with Neuralink. For example:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6914248/
(Elon Musk is author of this publication)
https://pubmed.ncbi.nlm.nih.gov/34140486/
https://pubmed.ncbi.nlm.nih.gov/33955717/
https://pubmed.ncbi.nlm.nih.gov/33894832/
https://pubmed.ncbi.nlm.nih.gov/33784612/
https://pubmed.ncbi.nlm.nih.gov/33236720/
A welcome upgrade to The Diving Bell and the Butterfly
https://en.wikipedia.org/wiki/The_Diving_Bell_and_the_Butter...
50 years from now, complete paralysis will be no excuse for not having a job. /s
Now imagine he had a cochlear implant with bluetooth that allowed for two way communication. That's telepathy.
First episode of Shark Tank[1] has you covered.
How did they get informed consent ?
From the article: "His wife and sister provided written consent for the surgery."
I’ve seen and built systems that are far less invasive —
https://agenda.hep.wisc.edu/event/792/contributions/18594/at...
With accuracy on par or better than described:
> He eventually explained to the team that he modulated the tone by trying to move his eyes. But he did not always succeed. Only on 107 of 135 days reported in the study could he match a series of target tones with 80% accuracy, and only on 44 of those 107 could he produce an intelligible sentence.
In fact, some of these successful efforts were done back in the early 2000s.
From the article:
> Researchers inserted two square electrode arrays, 3.2 millimeters wide, into a part of the brain that controls movement. When they asked the man to try to move his hands, feet, head, and eyes, the neural signals weren’t consistent enough to answer yes-or-no questions, says Ujwal Chaudhary, a biomedical engineer and neurotechnologist at the German nonprofit ALS Voice.
The speed and accuracy does take a hit using external nodes, but brain implants are frankly dangerous. They come with serious risks.
Does your less invasive solution work with fully locked in patients? Often these solutions rely (partially) on some tiny muscle activity still working, which they don't have. At least that was what I've been taught some time ago (could be early 2000s), and what's for sure hard to rule out in 'healthy' patients.
It was developed for fully locked in patients, who have eye control. It uses a very similar mechanism except it’s slower. Effectively you have to look at characters on a screen to type.
The article implies he has lost even eye-control, or at the very least, that he wanted the implant for use in that eventuality.
Are the risks associated with invasiveness fixed (eg, there will always be at least a 1/15000 chance that a patient dies within the first year of receiving a neural implant), or can they be reduced with better technology and more understanding of the brain?
At the moment, there will always be damage.
There’s also risks associated with the surgery (that’ll be much higher than 1/15,000; you’re probably talking at least 1/1000 just for risk of initial surgery).
Further, no implants last forever. They develop scar tissue around the implant. So you’ll have multiple surgeries if you survive years.
Technology can mitigate much of that risk, but we are pretty far away from removing risk. We also will need multiple major independent breakthroughs.