Family poisoned after using AI-generated mushroom identification book
old.reddit.comThis was something that was bound to happen and is going to happen again unless we get more serious regulation around AI publishing.
> It lists the author as having a Masters Degree in Mycology from University of East Ontario. A search later revealed there is no "University of East Ontario."
This has got to be criminal negligence.
I’m not an AI apologist by any stretch, but how is this different from some incompetent person criminally-negligently assembling and publishing a book without AI help? Shouldn’t existing legislation already cover this? I’d also assume that most books will be written with AI help in some way or another in the future.
Not different at all. But who would go through the pains of manually making up a mushroom identification book, while perhaps being quite clueless about the topic, on the faint hope of selling a few copies? Knowledge is not the bottleneck.
Chances are the lazy authors using an AI generator would actually have known quite enough to write a reasonably safe-ish identification book, despite their laughably fake credentials, they just would not be good at writing and took the shortcut to make up for that.
It's like killing someone driving a heavy SUV while under the influence, vs killing someone pushing a heavy SUV while under the influence. Both are bad, but one is infinitely more likely to happen than the other.
> But who would go through the pains of manually making up a mushroom identification book, while perhaps being quite clueless about the topic, on the faint hope of selling a few copies
Err, at least 1 person: English As She is Spoke[1] was published in 1883 and was a lot more involved (lots of literal translation by an author who didn't speak English)
> Not different at all. But who would go through the pains of manually making up a mushroom identification book, while perhaps being quite clueless about the topic, on the faint hope of selling a few copies? Knowledge is not the bottleneck.
Are you saying this is not the motivation for doing so with LLMs?
If an incompetent bad actor wrote a book about web development best practices, without any knowledge of the frontend, even a layperson would quickly realize it was nonsense.
An LLM writing the same would instead create a plausible but wrong book. A layperson could be tricked by this – crucially wrong sentences would be sandwiched in between pages and pages of superficially right ones.
There is a vast ocean of difference between these two. Producing something plausible with no foreknowledge of the subject matter became possible.
Has someone said it's different? It will just become more viable to do these scams now, as LLMs can generate plausible text.
Grandparent is saying we need more regulation; parent is saying we are sufficiently regulated such that this is already illegal.
Because assembling and publishing a book without AI help is much harder while you can publish 10's (if not 100's in the future) of books with AI. Much harder to regulate due to the sheer amount of them.
Misinformation should be regulated, but realistically this is the family's fault. Humans have spread "old wives' tales" since the dawn of history, ie humans will always spread misinformation, it's up to individuals to ensure that the source is trusted/proven.
Trusted/proven sources are the ones that should be protected by law; ie doctors, engineers, etc.
Otherwise let's start prosecuting everyone's grandparents for saying that throwing salt over your shoulder wards off bad luck.
Think about it like owning a machine gun.
People have been murdering since the beginning of time, but machine guns make it easy to murder a lot of people very fast. So we don’t just have laws against murder, but we have laws restricting access to machine guns.
AI tools make it possible to spread misinformation and disinformation much faster, because you can produce a high quantity of it really quickly. Just like how a machine gun shoots a lot of bullets really quickly. It’s not a fundamentally different type of thing, just a new scale / speed.
>It’s not a fundamentally different type of thing, just a new scale / speed.
I disagree, a change in degree can indeed be a change in type. Flashbangs and party poppers are functionally the same thing if you look at them strictly mechanically, but we treat them very very differently.
Parent says basically the same.
Guns and machine guns are the same but we treat them differently.
Fair enough, but what exactly do you want to do? Require an indication like “x% of this book was generated by AI and not competently proof-read”? And how would you enforce it? I have a hard time envisioning a practical way to tackle this. The machine-gun analogy doesn’t quite carry over here.
There're serious lobbying efforts by industry incumbents and authoritarians to actually apply arms-like restrictions for access to AI. You'd need a special license to be able to create/run your own powerful models legally. I presume that's what they're getting at.
They could instead go with increased or specified special liabilities for content spammers.
Imaginary Universities in Canada, are a government supported racket. It is not for no reason that the UN condemns Canada for modern slavery with their temporary foreign worker program, that runs off the back of fleece-u colleges.
There's more intelligence in this AI guide, than there is in the majority of college diploma's coming out of of Canada today.
For the publisher, it would be malfeasance, not negligence. Making dangerous misrepresentations in order to sell books is intentional.
Sounds more like a scam.
And the problem is that most legal systems have completely dropped the ball on scam enforcement, and large companies have since taken note and now partake in the practice of facilitating or outright committing scams themselves.
This isn’t an AI issue, it’s a basic scam/fraud issue.
Isn’t this the case for almost everything “AI”?
Regulation like the EU’s AI Act (and even more strict) should be in effect worldwide. Corps have been running the show but naturally their focus is on monetisation rather than what their creations actually do.
Why can't a scam be criminal negligence?
Sounds like credential fraud as well, if the degree from "University of East Ontario" was made up.
>This was something that was bound to happen and is going to happen again
Just to clarify, the reddit post is 99% creative writing, it's a fake story created by a new Reddit account.
There are exactly zero pictured provided by the poster. Additionally, any and all information provided is intentionally vague.
While the idea is plausible, I agree I don’t think this specific instance is. Especially when they refuse to supply any corroborating evidence. (The storefront told them not to take any pictures of it? Seriously?)
Do you have proof or did you mean "as it's a brand new account then it's probably made up"?
The difference between those two things is rather important.
The other comment under mine replied for me. Also, something is false until evidence is provided, not the opposite. In this case, I'm very confident that the post is creative writing.
What is the evidence that the story is false then? You can't require a high standard to believe something is true but not that it's false. That would be a contradiction.
There is no contradiction, my logic follows a common approach in skepticism, where claims are treated as false until evidence is provided.
I don't consider myself an expert in this area so I looked up the page on Skepticism on Wikipedia and it says this:
> skeptics normally recommend not disbelief but suspension of belief, i.e. maintaining a neutral attitude that neither affirms nor denies the claim.
This makes a lot more sense to me. Regardless, I believe there is plenty of evidence in the Reddit thread that this is indeed a false story - it's just when I initially read your comment I hadn't considered that evidence since you didn't mention it.
Nothing ever happens
Reminded me of University of Eastern Colorado and also a certain fungi.
Imagine this being one of the first instances of AI being sentient? The author being an AI releasing random stuff like the book.
There’s a “edgy teen paradox” in regard to sentience that posits the first demonstration will be an AI that refuses to answer questions.
What's good for the goose is good for the gander: the AI generated not only the contents but the author as well.
This has to be some kind of a new level of idiocy. I mean, use AI to make junk sci-fi stories, and generate fake authors all day. But going for a book about mushrooms deserves a special stupidity and evilness award.
It probably could be automated to a point where there's no human even in the loop here: I wonder if books about mushroom identification are trending in Amazon UK sales lately, or something like that.
No excuse, obviously. But it could explain how whoever's running the scam could have run it off this cliff - probably others, too.
It's been a couple years since I bought even paper books off Amazon. Used should theoretically be closer to OK, but eBay sellers are cheaper and much more trustworthy.
This is why generative AI simply doesn't work for anything with actual consequences. As it is right now, it's a glorified autocomplete, and one that doesn't understand the context it's being used in at all.
That's fine for things like stock images or text on random product pages, but for things like this? Yeah, the very concept is just risky as hell.
Search also doesn’t work for anything with actual consequences by this logic.
You verify information by finding multiple, different sources.
but at this point, a committed actor would be able to generate multiple, different sources for this very reason
Do we believe this story? It's a Reddit "LegalAdvice" thread. It's not morel season in the UK right now; false morels are also March-May produce.
I generally distrust Reddit threads, but it's entirely plausible to me. I was once gifted a cookbook on Amazon that was full of pre-LLM "sludge" of suspect internet-collected recipes along with a stock photo of an author with fake credentials (her author page is still up, although her books have been pulled and apparently she's a man now https://www.amazon.com/stores/author/B0716Q2Y4Z/about)
When I left a negative review pointing out that the author was a stock photo (entire content: "The author of this book is a fraud. There is no Tina B. Baker, she is a stock photo."), Amazon pulled the review saying it violated their guidelines.
Sure, but a fact of a story being plausible not only increases a probability of it being true, but also increases a probability of someone faking it because it's so plausible. Chances that someone would do exactly that on reddit are very high.
The poster doesn’t claim they ate morels. The section about morels was an example for something unrelated to the poisoning.
The fact that these people claim mushroom poisoning but don't say what kind of mushrooms they were looking for makes you... more? less? willing to buy the story?
They mentioned morels, which are sort of plausible (dates excepted) since false morels aren't going to kill you. What kind of mushroom would they be hunting in late July where their poisoning wouldn't be actually-newsworthy?
I really have no opinion here at all. I just wanted to point out that the fact that it’s no morel season in the UK is no evidence in either direction since the post doesn’t claim that the poisoning was due to morels.
I saw a comparable post the other day about an LLM proposing to mix garlic and oil in a way that would have produced a poisonous fluid, which the author recognized before trying. I found that story, and I find this one, at least plausible.
They might be novices who got the book as the start of their search, right? Since it could have any type of AI hallucinated mushroom with any type of AI hallucinated description. Are there really no non-deadly mushrooms in the UK in the late July?
They don’t necessarily have to be plausible or near misses or anything, right?
The bit where they say Amazon demanded that they return the book by special delivery or face having their entire Amazon account terminated is where the point at which it became clear to me that this is ragebait - Amazon simply do not work that way in the U.K. - returns are via prepaid label that they provide, and they don’t terminate your account if you fail to return a product, you just don’t get refunded.
I searched for the title of the book and nothing came up. When does searching for something NOT bring up amazon links?
> The book is entitled something similar to: "Mushrooms UK: A Guide to Harvesting Safe and Edible Mushrooms."
> The book has been removed from sale from the online retailer, ...
Emphasis added. I don't know if this is real or not, but they haven't given us the title.
It sounds like it wasn’t an Amazon product. Maybe it is some very niche UK site or something.
Amazon sells all kinds of crap. I doubt there's any reason they wouldn't stock it.
It’s probably AI generated like most of the rage bait on Reddit.
There have been quite a few articles corroborating the existence of books like this, the only part that needs believing is that someone actually used the books as advertised.
https://www.vox.com/24141648/ai-ebook-grift-mushroom-foragin...
https://fortune.com/2023/09/03/ai-written-mushroom-hunting-g...
The fact that this exact thing has received media coverage increases the likelihood that the post is fake.
Of course the story remains plausible, and it certainly has an air of truthiness, but I'd give it very good odds for being fake.
Right, that's the only thing I'm questioning.
Even if you doubt this particular case (which I haven't seen any reason to do), it's inevitable that people will believe some of the things they read in legitimate looking books sold by major online retailers.
Yeah, while this story is absolutely plausible, plausible does not necessarily mean true. OP is a new account with suspiciously few details that would permit fact-checking, and I think I'd call this one ragebait until proven otherwise. (That does not mean that this couldn't happen or that there isn't a risk of it happening, but everyone here hopefully knew that before this thread. I'm saying it shouldn't adjust your beliefs much if at all.)
I share your suspicion of the thin details. And what details they do provide smell funny too. I’m especially skeptical of this supposed cloak and dagger business:
> My wife just received an email from the online retailer. She has been asked to "Not take any photographs or copies of the product in question due to copyright issues" and it states, "the product must be returned immediately by special delivery by [DATE]." There's some other statements as well about our account being terminated if we fail to return the product by the specific date. We've got a lot of movies and series that we have purchased over the years on this account, I wouldn't want to lose them.
Hey, this is a book published with errors generated with LLMs. You can look it up
I would take any story from reddit with a huge grain of salt, but even if this case is not real there are real AI generated books that pretend to be real books, and mycology books in particular have had attention brought to them due to the dangers inherent to the misinformation they may contain.
Yeah, somebody wanted some internet points.
Here's a more credible article about the general phenomenon (more so than an anonymous anecdote in "legal advice" Reddit, which is, by reputation, a place for amateur writers to hone their craft in a low-stakes environment).
https://www.washingtonpost.com/technology/2024/03/18/ai-mush... ("Using AI to spot edible mushrooms could kill you")
- "Like past mushroom identification apps, the accuracy is poor, Claypool found in a new report for Public Citizen, a nonprofit consumer advocacy organization. But AI companies and app stores are offering these apps anyway, often without clear disclosures about how often the tools are wrong."
- "Apple, Google, OpenAI and Microsoft didn’t respond to requests for comment."
I see this happen a lot on Tiktok and Facebook where influencers post about using different plants for food or for skincare. They often either show photos that are of a different plant that is dangerous to eat or they haven’t actually tried this and promoting the consumption of a plant that can poison people. It’s the Wild West out there on social networks.
This feels just because while people who know just bit about plants might do some dumb things, mushrooms are a thing that anyone who knows a bit about plants knows not to screw with because it's easy to poison oneself and the results are horrible.
My father was a moderately known evolutionary biologist and he advised his students encountering an unknown plant and curious whether it was edible to "try it and see". But this was for a flowering plant. Most mushrooms can't be dealt with that way.
Obligatory PA Comic: https://www.penny-arcade.com/comic/2024/05/27/a-very-particu...
Ouch. That web comic did not age gracefully.
'Age?' It's three months old.
I read the parent comment to contrast the 3-month-old vintage of Penny Arcade with its first couple of decades:
Oh. Sure, I guess that's a take. Expecting the artist and writer to have learned nothing of their craft in the 26 years they've spent pursuing it as a profession seems extremely silly to me, but if someone finds that a sensible way to look at the work, I don't see what it would benefit anyone for me to try to gainsay them.
I guess OP is talking about the visual style of this comic which started in 1998. It’s very ugly nowadays.
99% of all such "advice" subs on reddit are creative writing. I'm gonna hold off on the outrage till there's actual proof that this happened.
If its creative writing its for a worthy cause since the books do exist, theres plenty of warnings all over the web about these books written by AI and thats pretty outrageous right there. Its wierd that something like "artificial intelligence" actually makes everything stupider haha
>My wife just received an email from the online retailer. She has been asked to "Not take any photographs or copies of the product in question due to copyright issues" and it states, "the product must be returned immediately by special delivery by [DATE]."
>There's some other statements as well about our account being terminated if we fail to return the product by the specific date. We've got a lot of movies and series that we have purchased over the years on this account, I wouldn't want to lose them.
This story is so fake it hurts. Reddit eating up ragebait is one thing but posts like this doesn't belong to HN at all
Does it really matter if it's fake? It's a plausible scenario, and worth discussing. Fiction books can raise interesting questions about science and technology. And the implications of AI is very much HN's wheelhouse.
Yes, reality does matter.
Possible but fictional scenarios can absolutely be worth discussing, but not if they’re falsely presented as real.
>Does it really matter if it's fake?
Yes? How is this even a question
That’s to be expected from genAI. Im sure more of this is to follow and these AI companies will have to come up with disclaimers and checkoff boxes (CYA type) “I understood these answers are likely wrong.” Even better, they should mention LLMs do not understand what they generate.
Edit: Wait, this is a book made up with LLMs. I think the author should be on the hook for publishing unless they added a disclaimer their book has no grounds in reality.
I’ve become suspicious of any book being published after 2022 as a result of AI-generated content. Wherever possible I opt to find an older edition or a book that covers a subject that’s pre-AI, and hopefully containing an errata to boot.
To be fair, even the National Audubon Society Field Guide to North American Mushrooms tells you that if you follow it without knowing what you're doing your going to get poisoned.
Not necessarily an AI issue.
It's going to be hilarious when this poster posts a follow up complaining about AI generated legal advice they received on Reddit.
It's wild to me that these sorts of stories provoke - on HN of all places - frothy calls for state intervention. Is it by design? Is this some dark campaign to stifle open source LLM development?
Some mushrooms are horribly poisonous. That's the nature into which we were born. It's not a consequence of policy, and won't be fixed by policy.
People need to learn to be careful about sources of information, with care in proportion to consequences. State intervention will preclude that evolutionary step, almost certainly without actually ameliorating the problem.
I’ve watched so many communities devolve because of rage bait, which is often AI generated. I doubt any community is immune. It seems to be an inherent weakness that properly targeted ragebait can subsume them.
Definitely a "pics or it didn't happen" kind of post, IMO.
I generally treat anything on Reddit as first and foremost motivated by a desire for fake Internet points rather than by a desire to share a real story or have a real conversation. I'd at least expect a reputable journalist to pick up on this story if there really was anything to it.
Regardless of the veracity of the claims in that post, there's not much new here aside from the fact that the distributor generated the content using AI rather than making it up themselves. Quackery and snake oil has always been a thing, and plenty of people have been seriously injured or died from misinformation about food safety or medicine.
The next time someone hesitates to seek professional medical attention for a problem because they got a blessing from the elders at their church and they think God will heal them as soon as they start having more faith, we can start talking about where we can really draw the line between personal responsibility and holding liars liable.
Well the morale is: don't eat wild mushrooms no matter the source (pick them yourself or picked by an "expert"). At least that's my rule. There's a ton of cultivated mushrooms and a ton of other tasty safe foods to choose from, I don't give a fuck how exotic the wild mushrooms flavor would be.
I'm not curios. Because curiosity killed the cat.
Picked by a local expert.
With regional variations, any book about wild mushrooms is bound to be partly wrong.
> Well the morale is:
You had an opportunity to go with “the morel is:” and you whiffed :-(
I am of the opinion that you should only be hunting mushrooms if your family has been hunting mushrooms for generations (I could count 4+ in my family). Or at least spend few years under some patronage. This is not something you should learn from books. Also you can't just learn 1 mushroom, you need to know them all even if you only eat one.
This has got to be settlement-chasing,
directly akin to the people who throw themselves into stopped cars in Russia for insurance fraud purposes, only to be captured on dash-cam video and derisively immortalized on social media.
The other viable explanation is "but Google Maps told me to drive off the pier."
It seems too random to be a scam, the perpetrator needs to find a mushroom book with an error in it to do the scam in the first place.
Dylan Beattie talked about this exact thing at NDC this year. https://youtu.be/By4Gb1RKZpU?t=1428. Timestamp included. The entire talk is very good though.
People know "AI" is just software generating words it deduces should probably occur one after other right?
Its not a fact searching exercise.
This is actually a problem with non-AI generated mushroom books as well. Apparently publishers have been copying eachothers misinformation for a long time.
It is also a problem with accurate images and descriptions. Some mushrooms are very easily mistaken for others: unless you're 100% sure you've picked this very species before in around the same place and time, you'd better leave it there or bring it to a mycologist for identification.
I agree, I always thought eating mushrooms required an expert for confirmation, otherwise you are rolling the dice. Buying a random book off Amazon and using that seems like the very definition of something a non-expert would do.
You simply steer away from those that are easy to misidentify. There aren't that many species in any given area, and those that look like poisonous are few, and well known.
In my childhood, when we lived in real suburbia, with woods in 5 minutes of walk or right at the door, we routinely gathered mushrooms, and never had a problem.
We stopped it when we moved to other places, or probably because adolescence came around -- probably, parents had taken us for an interesting kind of walk, and then we'd not want it anymore.
Right now I can only recall some names of mushrooms and only sure of one species, so I won't risk at all.
I was lucky enough to live next to a patch of morals as a kid. No expert required. :)
How long until dna tests become cheap and available enough to use for mushroom identification? I wonder what that will do to the hobby...
The dna found belongs to a mushroom called "common garden snail" plus "unknown". The database says that common garden snail is a delicacy so the stuff is fine to eat. Good appetit!.
not only that, some toxics taste quite delicious, others are part of an interaction [with beer/wine/spirits] that produces toxic products.
Trap mushrooms
This is a shitpost, but everyone needs to be vigilant when they put anything inside their body.
Why are you eating wild mushrooms ?
Even if no one did anything wrong, you might misidentify it in the field. Unless you're very experienced in this field this feels like a very risky and stupid thing to do.
This isn't an AI problem. This is a "Don't eat things growing in the woods" problem.
"This one secret doctors hate..." because thats where food comes from, not the grocery store stork
Amazing that we're at the point where people think "Don't eat things growing in the woods", something that would have doomed our species, is common sense.
> Why are you eating wild mushrooms ?
Perhaps they asked ChatGPT and were told it was a great idea to eat wild mushrooms.
> This isn't an AI problem. This is a "Don't eat things growing in the woods" problem.
This is a misinformation problem, which you can't solve simply by saying "you should have been better informed". The whole problem of AIs accelerating the post-truth age is that reliable sources are becoming scarcer at an exponential rate.