Behind the scenes with the Midjourney scanner [video]
youtube.comThis skepticism is weird to me. I might be overly naive, but it looks like these are people that have funding for doing something they think is cool and think could help people out. Whether or not it turns out that way is something we'll have to see, but the premise of just going ahead and building the thing should be applauded in my opinion
I wouldn't want to get into a medical scanner built by an AI image generation company! Marketed as a spa treatment. There are so many things that can go wrong and the unseriousness around the whole thing bothers me.
> There are so many things that can go wrong
Are there? My impression is that basically the only thing that can go wrong with ultrasound is they pump too much energy into you, and as risks go that seems both difficult to screw up and easy to make sure you don't screw up.
The imaging might be useless, but then that's why it's a spa treatment. I assume the primary use cases for this are (at least until it's developed a lot more seriously) "cute" non-medical baby images and body composition - the former of which can't really go wrong and the competition for the latter is a scale with some electrodes making shit up anyways.
I think the negativity is due to the vibes and framing of the "spa" for rich people to hang out and do some low-key "health optimization", it ties to Silicon Valley longevity stuff, pattern matches with the vibe of cryogenics and other quantified self stuff etc, instead of a vibe of making something that broadly improves the health of the masses and lower classes.
As a business case that looks great. Somewhere between genius and standard practice for most industries.
Start with low-volume runs targeted at upperclass customers who can afford the pricetag. Bonus points for a group that has a proven track record of spending unreasonable sums for unproven technology. Over time the user story, manufacturing, parts sourcing, regulatory approvals all get sorted out and you can move down into bigger and bigger markets and lower and lower price points
Of course that risks being associated with all the crap the same high-spenders spend their cash on
I agree. It's very clearly a research prototype and they are talking about getting it working to do body composition.
There's a risk of echos of Theranos here. A paper apparently describing this ultrasound approach has been uploaded to arXiv [0]. If so, the resolution demonstrated is nowhere near sufficient to detect small changes to anatomy, let alone monitor them over time. Future developments could obviously improve on that.
[0] “Whole Cross-Sectional Human Ultrasound Tomography” https://arxiv.org/pdf/2307.00110
They don't make any specific claims about what conditions it will diagnose. At 16:30, he says they are only initially doing "body composition" because anything more would add 9+ months to the deployment timeline. I assume they mean they process the images to return an estimate of body fat/muscle mass. Which isn't difficult and it seems likely they could get the error bars pretty low just off estimating subq fat alone. He doesn't say the specific classification they received from the FDA, just that it is a class 2 medical device.
Just reliably estimating body fat and muscle mass could already sell well with high-class fitness studios. Those already offer those scales with handheld probes as a feature, and those contraptions are horribly inaccurate. With this you not only get accurate estimates, you can also attribute them to your body parts.
"This month you gained 5% muscles mass in your upper arms, but lost 3% in your core muscles" "Body fat in your upper thighs reduced by 10%. You are on track to reach your goal in 6 months". It might be expensive, but there would be plenty of customers willing to pay for that
Then they can work on getting price down and accuracy up, both of which seem to be goals for the future
Stay tuned for new versions of the scanner! Hopefully later this year. The physical limitations are still far beyond all the current implementations. We'll see how far we can push it. If it's not far enough to be useful then at least we tried :) but I'd be surprised if it isn't useful so that's why im betting our own money on it.
one of the lead paper authors (jinhua xu) works at midjourney, appears in the video, and comments specifically on how the midjourney approach is the next, significantly better-funded iteration of the paper approach
If you watch the video, the original prototype had hand built piezoelectic array, which was a complete pain and nowhere near as good as the current revision. This one uses COTS hardware, just lots (40+) of them.
true, but he doesn't say if it will meet the threshold of 'useful'
I have a similar feeling.
One the one hand its great that they are spending the time and money to do this, on the other hand I am _very_ suspicious of their motives.
Getting _a_ picture is not that difficult, getting an accurate, repeatable, high resolution picture is a lot harder, and state of the art.
my worry is two fold:
1) over promise and causing injury to desperate people who see smudges on scans and have invasive surgery only to find out that its a reflection/artefact
2) what are they doing with the data they collect, and how will it be used to make money.
I think the main issue is that there are "no good startups" any more. As soon as an innovation happens that might be worth something, your original CEO is replaced by someone driven entirely by money, rather than public good. Or they get bought out by a corp that only cares about maintaining a monopoly.
In the video David says he's building it just because he wants to. It really sounds like his own passion project. No investors. It's a nice position to be in.
Now I don't know this guy in the slightest. One view could be he's just a geek like us living out his dream building cool potentially useful stuff not entirely sure where it will lead.
A cynical view is it's all about $$$$.
Either way it's great HN content!
I should have made clear, my suspicions are not really around the engineers running this, its the people who are less motivated by the original cause that can see $$$ in it.
Thats the point right, most of us would, if given the chance, divert some funding to work on a passion project that could easily save lives. That I applaud and would love more people to be able to do.
We have no investors, we could pocket the money. We're choosing instead to spend it on projects like this <3
>Getting _a_ picture is not that difficult
Can't wait to read your papers and/or watch your videos where you show us :).
https://thepihut.com/products/ultrasonic-distance-unit-i-o-r...
as I said, an image is doable, there is opensource to do that: https://hackaday.io/project/159467-open-source-ultrasonic-ph...
accurate and repeatable is that hard bit.
There's one GIGANTIC difference between Midjourney and Theranos.
Midjourney's money is their own. They don't have to lick anyone's boots (or worse) just to put bread on the table.
Don't ever confuse actual innovators with low-tier VC scammers. Because of that, I'm massively bullish on them.
This definitely isn't another Theranos. Theranos claimed to have a blood test that didn't actually exist. This is "just" standard ultrasound but with a much wider aperture than normal. There's no new science, it's just engineering that nobody else has put the effort in to actually do.
It’s literally a little fun side project for them and it’s in no way attempting to make billions by manipulating results. They straight up show and tell the limitations of what they are doing. Theranos was a complete scam from the get go and lied all the way til the end. I don’t see a single similarity
One thing that’s kinda awkward in the video: they mention one of the big shortcomings of ultrasound being that it can’t image “airy” organs like the lungs, and their expert responds to that by mentioning that the amount of angles/devices means that you still get imaging of everything surrounding the lungs.
But the critiscism isn’t that the lungs would obstruct you from imaging certain areas, it’s that there’s just very salient parts of the body that you can’t really image with ultrasound, which means this would not be a full bodyscan even if the resolution was incredible.
I think there’s some genuine intent here, if for no other reason than that it seems silly to transition from ai to hardware if you’re purely trying to grift. I just wish they responded candidly to the obvious questions people have.
Every imaging modality has pros and cons. Like CT imaging is bad at soft tissues but much better at bones than MRI. This physics has a lot of really cool and unique properties and imho it obviously makes sense for someone to invest in them. And we are (with our own money, no investors, etc). In my mind, in the future, you really want the best "megabytes per second per dollar" about your body. And it's hard to beat this by that measure. It's also very possible to keep adding more and more sensors to this system. I'd love to add some more that further improve the imaging inside the lungs and brain. We'll see how far we get but imho over the long run we'll be the fastest way to do a lot of things and it will help a lot of people.
Not being able to image inside the lungs is probably only a minor limitation really. There's also inside the head, and inside the rib cage is going to be awkward too due to bones.
Also ultrasound just isn't that good of an imaging technology, even with full aperture.
That said, it's non-ionising and if they can make this reasonably cheap (big if), then it's better than nothing at all. Probably decent for finding cancer, especially breast cancer (no pesky bones there!).
> it's non-ionising
True, but ultrasound isn't entirely benign. Depending on what frequency and what power it can crack solid parts of the body (its used to break up kidney/gall stones) it also can heat things up (which is, assuming my understanding is correct, why doppler scans for blood flow are not done on <18 week pregancy)
The probes they're using are off the shelf probes used everywhere including on small animals, I don't see how you could compare that to a lithotripsy
lasers can cut steel, so barcode scanners aren’t entirely benign
Yes, no Reductio ad absurdum.
My point is, phased arrays of ultrasound have the ability to be destructive. In the same way that a single MIMO wifi AP isn't going to cook you, but 10,000 MIMO arrays steering coherently over you will.
You know what already detects breast cancer? Mammograms followed by ultrasound.
We have screening programs for a reason. We know the sensitivity and specificity of these. They are widely available in any rich country that doesn’t treat its citizens like shit. There will absolutely be better stuff out there as we progress, with better sensitivity/specificity and lower harm (everything medical has some harm quotient) but I have a hard time wrapping my head around how they will best physics to provide better than state of the art today with this technology (and bow they will beat availability of current screening systems)
https://en.wikipedia.org/wiki/Liquid_breathing ? So totally solveable
I dont understand though, why you have to simultanously do this from all sides- have the ultrasound swim around with the patient? Takes out the comlexity?
Or use boundary layers to keep the sound on the slice? Turn this cocktail glass full of patient into a tequila sunrise?
I don’t know if you’re joking, but liquid breathing has only really been tested on rats, like famously the one from The Abyss https://en.wikipedia.org/wiki/The_Abyss, which died a couple of days later.
> I dont understand though, why you have to simultanously do this from all sides- have the ultrasound swim around with the patient? Takes out the comlexity?
One sensor can only see waves that reflect straight back. If you have more sensors (and fire them sequentially, like they do) you can detect pulses that are deflected, and measure the deflection angle. That gives you much better data
I get the impression that what they want is real-time scans, so you can see the organs moving and everything. But that would require a 3d arrangement of sensors, which is too expensive. Instead they have a 2d arrangement and move the sensor relative to the person in 1d. Having only one sensor and moving it in 2d would harm scanning speed and fidelity even more.
As I understood it, they're using the rest of the array to receive when one is transmitting, thus getting signal from the opposite side of the body as well. I don't see how you can do this with just one device.
You are correct. I was wrong. Thanks
You’re suggesting flooding the lungs in order to get a scan?
Does this whole thing seem fishy to anyone else?
More dolphin-y than fishy, it's ultrasound after all :)
The announcement video has some quality that made it hard to shake the feeling that I’m watching a black mirror episode. Or a sequel to Ex Machina.
Yes. We are right to be suspicious due to the main goal of start ups and corporations: profit for their shareholders. Theranos will always be in the back of my mind regarding healthcare technology start ups.
We have no investors! We're doing this with our own money because it seems like a good thing to do :)
They hallucinate images for a living, of course making a medical imager is fishy (to put it extremely nicely).
There's no AI in anything right now, we're just using physics based classical algorithms. It makes sense to eventually use AI but in the context of like initializing a physics solver to reduce the step-count.
silicon valley startup caricature in the style of a youtube influencer, glossy, moving picture, edgy, narcissistic, hyperunrealistic
getting serious theranos vibes from this
Theranos was claiming things that would be incredibly useful, with technology that doesn't work. This is claiming things of uncertain usefulness, with technology that does work
The common thread is commoditization of medical technology to enable more frequent and useful testing. But 3d ultrasound machines exist and Midjourney's setup looks completely plausible. Their big bet is that it's actually useful in medical applications beyond better baby pictures
Good to see Mr. Valente is doing well
The new scanning technique is very fancy, but it sounds like people might be better of with boring full body visual imaging to check for skin cancer. If they want an AI angle to that, Midjourney could easily make a digital twin from the scan.
How is only scanning the outside better than checking the inside?
Sure, visual imaging is cheap and high-definition, but it's also a pretty low value product. People can already see the outside of their body. Instead of a fancy full-body visual scanner just give them a pamphlet which things are normal, which are abnormal but harmless and which need a doctor visit.
Seeing the inside of your body on the other hand opens up a lot of things you previously couldn't see. We have to be careful not to overreact and try to "fix" all bodies to fit the image printed in the textbook. But the potential to do real good is huge
I’m just wondering when people are going to realize sticking your body into water being vibrated with ultrasound isn’t a good idea?
I used to have ultrasonic cleaner for jewelry and one of the things advised is to not put your hand in the water when it’s vibrating as it can be bad for your bones.