AI is destroying the trust between writers and readers
theatlantic.comThe year is 2110, and two human writing powerhouses are pitched against each other in mortal confrontation. In Alexandria, Egypt, there's Writers' Street, where both seasoned and aspiring novelists type stories at street cafes, using self-invented codes in chromatic scales that they pluck on their guitars, in front of tourists and ubiquitous survtech that unwillingly--or not so much--witness. Others simply sing their tales. Not for the faint of hearing.
Off the shore of Italy, in Montecristo, the Benedictines have regained full control of the island. No Internet or AIs are allowed there, but their hospitality business rents mechanical typewriters by the hour. More than once, a famous writer has jumped from the island's cliffs, unable to bear the shame of being accused of using AI to produce their magnum opus.
For the unwashed masses, AI-produced literature is free and abundant. The discerning, however, have gotten a taste for manually bound tomes exclusively inked with the output of human brains.
(you possibly had no trouble guessing it, given where those places are, but just in case, this came from a column in themediterranean)
I just started reading infinite jest and I can't help, but wonder if that is what is coming.
enjoy the ride, at least twice. that's a book that bears rereading
But the worst of it was, all the third-rate poets emerged unscathed; being third-rate, they didn't know good poetry from bad and consequently had no inkling of their crushing defeat.
Maybe we could devise Vogon's simily enchancer?
edit: allegory augmenter
"AI is forcing a re-configuration of the ways trust is established between writers and readers" --> there, I fixed the title
There are many ways that I think what we are currently referring to as "AI" is disrupting existing relationships in society.
To the extent that that disruption is forced by external, pointed, injection of capital (e.g. russian bot farms, heritage foundation bots, etc.), it's pernicious. It has undoubtedly made those bad actors more powerful.
But, to the extent that that disruption is the result of asymmetrical power relationships (e.g. the outsized influence that publishers have), I think it's welcome.
I know who writes the things I read (because almost everything I read is over a thousand years old! but, that's beside the point...) because it's important to me to know who I am connecting to through the written word. I doubt my relationship to writing will ever be disrupted by AI because it's so important to me to know the history of the people whose ideas I'm consuming.
GenAI that imitates human art is the definition of an anti-invention. If an invention is a new innovation that improves something for the human experience, an anti-invention is a new innovation that makes life worse.
This is just one of several areas where GenAI has been harmful and should start getting regulated like digital drugs.
I don't think that definition meets what people would consider an invention.
The AK47 is an invention. The inventor "would prefer to have invented a machine that people could use and that would help farmers with their work - for example a lawnmower", but people still consider it an invention. Inventions do not have a polarity in the axis of quality of life.
There is no consensus on if GenAI is overall harmful or not. In general people are optimistic but concerned.
The last thing we need is another war on drugs.
I would rate A/B testing on human behaviour, or advertising to be more harmful to the overall human experience than GenAI.
“What about guns and bombs” is not a counter-argument that matters much, because you have to consider context. We can get into the weeds: guns and other weapons were used by various liberation movements to free the world from fascism and other forms of tyranny, and also to kill feral hogs attacking humans. The only real use for AI novel generation is if you’re stuck on a desert island with nothing else to read. But such a scenario is absurdly unrealistic and served better with a library full of real books. Anything an AI novel can do is done better by a human novel, without all the harms. Whereas a gun is the best way to kill a feral hog. A gun might be an anti-invention, but AI art generators are definitely anti-inventions.
Guns makes lives better if you are the disadvantaged and is at the receiving end of the abuse by the powerful. It makes the playing field even for you, which makes your life better..
I think that, in the history of the world since guns have been invented, they have mostly been used to the advantage of the powerful, not the powerless.
There's a quote about that by Hillaire Belloc:
https://www.goodreads.com/quotes/355429-whatever-happens-we-...
yep - guns are a guarantee of liberty.
No, courage and community are. Arms are only the vehicle.
Damn can't believe Samuel Colt is on HN
> Guns makes lives better if you are the disadvantaged and is at the receiving end of the abuse by the powerful.
...only if you have the gun, you know. Which the disadvantaged people tend not to. In fact, let's generalize your statement a bit: "Tools of power and coercion make lives better if you are the disadvantaged and is at the receiving end of the abuse by the powerful". No, they don't! They do the opposite for very obvious reasons.
> There is no consensus on if GenAI is overall harmful or not.
At least in part because it does not exist, oligarch marketing notwithstanding.
https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
You likely are referring to plagiarism slurry.
There is a mistaking of nomenclature here, but it may be purposeful by the tech oligarchs. Artificial general intelligence is usually abbreviated "AGI". The "GenAI" shorthand is usually meaning "generative AI", but it may have been coined intentionally as a nod to supposed future general intelligence built on the same technology.
generative, not general
Delegating the thinking and feeling parts of the human experience, mechanizing human creativity itself, is a direct attack on human dignity.
It makes people dumber, less self-sufficient and (maybe worst of all) less interesting.
>Delegating the thinking and feeling parts of the human experience, mechanizing human creativity itself, is a direct attack on human dignity.
And I'm honestly sad for anyone who disagrees because it indicates they don't have a deeper relationship to art. For me, art is fundamentally about communication. It's someone sharing a little bit of themselves. AI might be able to produce pretty images and sounds, but it has no experience or feelings so the art it creates lacks that magic of a shared human connection. Pardon the crass analogy, but it's like if someone said "I don't need any intimate partner, I have internet porn". It's a depressing mindset because it indicates that people can't even see what they are missing.
> so the art it creates lacks that magic of a shared human connection.
I keep hearing this point, and the thought that comes up first is 'but blood diamonds'. Let me explain, you can buy a cheaper 'inauthentic' lab-grown gem that's chemically/physically identical to a (nearly hypothetical) flawless diamond, but we buy the 'natural, and imperfect' diamonds at a much higher price. The blood of the orphans that pulled the diamond rough from the minds soaked into the diamond raising it's value. The value isn't the blood, but the effort it took to produce that diamond, the story of how hard it was to mine is the value, not the end product (a shiny rock), which in the end, is all anyone can see.
The second thought I had was that what people find meaningful, beautiful, important varies wildly, to the point that landscapes are meaningful. Meaning can be found in anything, regardless of the existence of intent.
The third thought I had is communication does not survive the translation of art unscathed or unchanged. There is this disconnect, between the artist and the viewer. The viewer only ever sees a finished product. The viewer extracts meaning from the work entirely without the author/artist's involvement. This isn't communication, this is a soliloquy, frozen in time, the artist can not expand, explain, or sharpen (except in another works) those ideas once released into the world.
This disconnect is the rub. There are no 'magic' pixels that dictate secret meaning. There is no 'human color' that indicates intent. Meaning itself can't be painted. What relationship can ever take place between artist and viewer through those works alone. It's a 1-way conversation. And considering my second though, that intent isn't even needed to extract meaning, you can see the apathy and rise of the use of AI in art.
The point of making art isn't to make money, or to impress other people, or even change other people's thinking. It's to express yourself. That can't be taken by AI.
The point of viewing art is to reflect on how you perceive it. That's true regardless of the artist/author, and regardless of the blood (effort) (or not {AI}) used to craft it.
Well, Duchamp's fountain is a great illustration that "art is fundamentally about communication." does not entail that a cultural/artistic artifact has to be created by the author at all. I as well can create a short story with help of an LLM and say - this is my vision and what I always wanted to say.
I can disagree with "direct attack on human dignity" while agreeing with "art is about communication".
I guess I could have been clearer on the connection between the original comment and my reply. In my opinion, the "direct attack on human dignity" comes when someone starts projecting the depressing mindset I described outward. To continue the analogy, it's not just being satisfied with internet porn for yourself, it's telling someone else "you don't need an intimate relationship because you can always access internet porn" and I think it's fair to categorize that as a "direct attack on their human dignity".
> AI might be able to produce pretty images and sounds, but it has no experience or feelings so the art it creates lacks that magic of a shared human connection.
Who cares? I want it to do my taxes for me, it doesn't need to be Picasso
>I want it to do my taxes for me, it doesn't need to be Picasso
It sounds like you are commenting in the wrong thread then. This article and thread is about what happens when people try to make it Picasso (or experience Picasso's work for them.)
You're right, I forgot which thread I was on
Computers have been mechanizing people's jobs away for god knows how many years now. Everybody on this site is guilty of it to some degree. I'm a doctor, and automating me out of a job has been one of the wettest dreams of programmers since the expert systems of the 20th century, and that dream is palpable in every HN thread about medicine.
But now that computers have come for the "creatives", it's suddenly an attack on human dignity? No.
Programmers are not special. Artists are not special. Humans are not special.
> Programmers are not special. Artists are not special. Humans are not special.
Really winning me over with this take. Definitely prefer this vs checks notes human experience, creativity, and dignity.
I was never going to be interesting or truly self-sufficient, anyway. As for dumber? Sure, I guess, at least about some things.
I’ve seen lots of AI-generated pictures and videos that I found insightful or funny. Also lots in bad taste. It’s not black or white. Like any other technology it can be used for good or bad.
There's art and there's taste. People with good taste may now be able to bypass the artist by getting the AI to create something. In some ways it's equivalent to the product person being able to bypass the programmer.
For art, "good" is always at least partly subjective. As such, your "good" might not be the same as my "good". It seems reasonable, or at least not morally wrong, to admit no good use of the technology for one's personal taste.
What the comment is doing is assuming no good use for absolutely anybody. That is patently wrong.
Why does a government have to be a parent? This feels short sighted.
I think there are other digital technologies that are also a lot like drugs and deserve similar regulation. I.e. if your social media company decides it wants to be a platform and insert stuff to inflame you or generate more engagement....it's a drug, not a tool for social connection..
I think there are ways you could serve ads and make money as a business, just not as much money, and not be basically a drug. The social media giants choose not to do this of course and will have to be forced.
I am distinctly less libertarian/classically liberal/etc. than the average user of HN though. So I understand that this opinion is likely to get me downvoted. I prefer to think that I live in the real world and have seen the damage these things have caused and am not in a tech-only friend group bubble.
Social media still connects real humans to real humans (sure often badly and trying to profit from it at the expense of users). This however takes the other human out and plugs you into itself, which is a simulated average of humans based on their work taken without consent
No it doesn't[1], except perhaps by turning a bunch of thoughful humans into an online mob. Social media amplifies humanity's worst impulses for clicks. The connection part is purely incidental and is going away now that AI can just generate whatever 30s conspiracy theory video you want to watch instead of having a person do it.
The connection thing maybe was incidental before but it's now the main value proposition of social media. we all can now generate sh#t by ourselves so why would we need social media at all if not connection. I have dozens of real people i know whom i keep in touch with through social media
unfortunately humans have shown themselves to be completely unable to properly restrict manipulative drugs like alcohol, caffeine (which doesn't need to be in sodas at all, the sugar is already addictive enough), nicotine, advertising, social media, and so on when capitalism would suffer as a result... we should not get our hopes up too high.
I think genAI has been strictly worse for humanity. It's destroying the quality of our industry, it turns creative fields into slop, etc. It is all downside and no benefit, in my opinion.
I guarantee you, good-quality AI generated art you see every day without noticing it.
The last season of Poker Face was really bad, especially the last few episodes. Come to find out Natasha Lyonne is big into AI and I have no doubt they had their AI slop out the plot and writing for the last few episodes. The writing went off a cliff, it was really, really bad compared to the first season. The show ended with me thinking WTF did I just waste my time watching? I have no doubt it was AI slop.
> Amazon spends a lot of time and effort trying to prevent these [fake] books from getting posted,” Mary Rasenberger, the CEO of the Authors Guild, told me, “but it’s always a game of catch-up.”
Do they really? They take a cut; they have incentives to let people get scammed.
Probably the same amount of effort spent on trying to prevent fake SSDs and usb drives.
Paraphrasing as I've not uploaded a book for a while, but Amazon has a button for "AI was used to help create this cover or content" I wonder how many authors actually tick it? I'm guessing none.
sample size of one, but I cancelled my spotify after the non-stop AI music slop.
Not out of principle, but because it's so incredibly distinctive and annoying. There's the composition that's bad (it's like all soul and gospel which was fun the first few times but got really old), but also the actual sound quality on a lot of AI generated music is awful.
Same thing with anything that's AI written. I just can't do it. It's like really bad fanfiction where someone tries too hard with forced metaphors and really stitlted pacing.
How do you use Spotify to get exposed to AI music? An honest question. I mean, I'm sure it's chock full of it but at least my usage patterns don't expose me to it.
Also interested by the answer!
All the playlists that Spotify generates for me are made of songs from actual humans and matches my taste.
The only times I'm exposed to slop is when I listen to generic playlists from Spotify or dodgy sources.
> but also the actual sound quality on a lot of AI generated music is awful.
I feel like I don't quite know what to listen for here.
There are a couple sound-quality based tells that I've noticed, off the top of my head
1. Usually weird little "flickers" of sound - like how a candle gives you mostly constant light but there are little flashes of bright and dark. AI generated music tends to have little distortions scattered throughout like that. As the models have improved they're less obvious but still absolutely there if you try to listen to any individual 'instrument' or 'track'
2. very low dynamic range - loud sounds and quiet sounds are much more similar volume than you'd expect from most real music, and things that are supposed to fade or taper-off in volume are unnatural
Hi-hats. When testing sound quality, pay attention to the hi-hats. Once you notice the shimmer introduced by audio compression to a hi-hat you can't unhear it, and for whatever reason AI generated music replicates this.
>and for whatever reason AI generated music replicates this.
Its within the training set I believe. The AI is being trained off compressed audio
I was listening to a podcast last year (forget which, or I'd attribute credit) and one of the tells AI researchers look for when trying to determine if music is AI generated or not is the compression artifacts. AI likely has gotten better at hiding it since then, but AI would include compression artifacts on the final product because of the prevalence of it in training data.
Yes, it's a naive assumption that quality of the presented material doesn't impact consumer behavior, if they get flooded with loads of garbage books and people can't find good book they enjoy they'll stop shopping at Amazon and it doesn't matter if they make money off the slop. It's the same reason Youtube has quality filters for algorithmic feeds, if the stream gets too polluted with bad videos people will watch fewer videos.
It's exactly the same process that's causing people to avoid expensive purchases on Amazon due to the number of fake products that have infiltrated their supply. People move on if they're sold a lot of bunk goods from a place.
I don't know. For years Amazon has had a fraudulent item listing problem to the point where it's hard to trust them. They have made no effort that I have seen to solve it. I mean, why would they, their sales haven't declined.
So, I am skeptical books will be any different than all those knock-off drop-shipped items.
I often trust AliEx more than Amazon these days. At least one of the two isn't pretending not to be what it is.
Plus Amazon has a long history of screwing over sellers. TL;DR if your product is actually successful, they'll make their own "Amazon Basics" version of your product, then deprioritize you (or even ban you over an excuse). They do this a lot. There's a lot more than just this, it's just an example.
I trust Temu more than Amazon these days. Exactly what I ordered shows up, on or before the expected delivery date, and works.
I get NVMes from them now, 2.5G Ethernet switches, and lots of other random items for a fraction of a cost of playing the Amazon lottery.
NVMes, and they're not fake? Your courage is impressive.
Works fine. Around $40 for 256GB (not 250). It’s TLC, quite a bit faster on reads than everything else in that price range.
I'd say that's hard to judge from the top line sales numbers, their broad sales could remain fine even when they lose larger rare purchases if they're bringing in enough new customers that are fine with only buying the slop.
Sales may not have declined, but returns have skyrocketed. I return at least as much stuff as I keep. Sometimes they tell me to just keep the crap because they know it's crap and it costs them more for me to return it than to just refund me and let me keep the crap thing.
And yet look at all the non-book products being sold on Amazon that are all garbage. Its the same slop problem and has been for a while, but they don't seem to be addressing that either. I have to assume it doesn't impact their bottom line as much as we think it should, consumers don't (or can't) care as much as we would hope.
Does it really impact people that much though?
There are two categories of (physical) products I buy on Amazon.
Regular “brand name” items sold in pretty much any big box retail store. I have more or less the same experience with Amazon as I do Walmart or Target or wherever for that category. Over thousands of orders I’ve never received a fake or had an issue outside of very rare packaging issues like damaged boxes.
The other category is “trash I need faster than aliexpress” and it’s basically exactly as-advertised for that junk. I usually get the item in two days or less where I am, and the quality level is more or less the same as ordering direct from China. Just trading expense for quicker latency. Useful for that last component for a project or whatnot.
I get that I might not be normal though. If I went to Amazon and searched for “grow lights” and got presented with what you get via that search I’d be quite frustrated. But I tend to research exactly what I want for those product categories and never had a problem getting what I expected.
> Amazon lets individuals upload a maximum of six books a week, allowing for endless churn.
That's a book a day (give or take), per account. No human author is publishing a book per day.[0]
It's interesting to me how little the incumbent platforms for digital content seemingly have been able to do in combating the tidal, exponential shift in submissions.
I just spent a month developing a Shopify App, on the side, for fun. When I launched it a couple of weeks ago, I discovered an app doing the same thing as mine with an identical name had also been approved and released a week prior.
How do people think that distribution will change now that supply has increased (at least) tenfold? If the consensus on X.com is to be believed, it's all about personal brand. This used to be the case, but perhaps 80% of it. Is conventional discovery dead now?
[0] - If this includes organisations, like publishing houses, then fair enough. But realistically we should be able to distinguish actual authors.
> That's a book a day (give or take), per account. No human author is publishing a book per day.
Suppose you're a 50 year old human author who has published a book a year since you were 18, and would now like to start selling them on Amazon for the first time. Should they hold you up for multiple years because you're rate limited to uploading one a month?
If the tradeoff for removing millions of false positives is one false negative then yeah, that's worth it.
Except that's not the trade off, because the slop producers would then just create more accounts instead of uploading more books on the same account. Which is probably what they're doing even with the existing rate limit.
So all you'd be doing is inconveniencing the actual human who just wants to use one account for all their books. Of which there could easily be more than one.
Moreover, trying to justify things with ratios like that proves too much, because slop can give anything a ratio like that. They can generate trillions of books, any number. If you banned everything then you'd be banning millions of pieces of slop for every one human-authored book, but you'd also be banning every human-authored book.
What you need is the thing that minimizes false positives, because the base rate of true positives is now so high that even a negligible false positive to true positive ratio would swallow most of the real books.
The example is disingenuous. It's a tradeoff we have to make, and there's not many 50 year olds hoarding their masterpieces.
It has nothing to do with hoarding. There are lots of authors who have published works using something other than Amazon, and for each one that uses Amazon there is a first time they started doing so. Amazon didn't even exist 50 years ago.
Tell us you're a software developer without telling us you are software developer. The real world doesn't have to follow strict algorithms. The 50 year old with a long history of publishing books, who now wants to sell those books on Amazon, has provenance to justify his case for exception.
Amazon is a website that follows strict algorithms. And what "provenance" are you supposed to have that their support staff could verify over the internet? Anybody can set up a website with made up dates on it.
That's such an unreasonably niche and unlikely situation to justify letting the rest of your customers drown in slop
That'd be like saying a forum can't have rate limits for posts because someone may write 10,000 discussion topics over a year and post them all at once.
> That's a book a day (give or take), per account. No human author is publishing a book per day.[0]
Note that it is 2 per week per format. So if you want to publish an e-book, print book and audiobook of your title, you can publish 2 titles, not 6.
Still more than most humans will typically sustain but it does change publishing patterns.
E.g. on Kindle+ (formerly Kindle Unlimited) a lot of people would publish short stories or short novellas. You absolutely can turn around more than 2 short stories per week.
The changes may push some writers to change what they write or bundle up stories.
> How do people think that distribution will change now that supply has increased (at least) tenfold? If the consensus on X.com is to be believed, it's all about personal brand. This used to be the case, but perhaps 80% of it. Is conventional discovery dead now?
For books, I'm hearing there are literary agents who now won't even sign anyone unless they have a sizeable social media following.
Conventional discovery for books will get you a very slow drip. But Amazon massively reward you if you publish lots and often by showing each of your works if someone clicks on one, letting you link series, and letting you do giveaways that are actually quite heavily promoted.
This, of course, incentivises people without a budget for promotion to push hard to release often. E.g. shorter titles, short-stories/novellas.
And that as noted above is being affected by the reduced limit.
> [0] - If this includes organisations, like publishing houses, then fair enough.
It includes anyone who is not large enough to get Amazons attention to get access to send them feeds directly.
I have a client that runs a small press, and basically their only option for Amazon is going via aggregators (and that too is relatively limited) and only pushing their top titles direct as until/unless they sell a lot more than they do now it seems they won't even get to talk to someone with any power to even escalate to anyone with authority to let them bypass KDP.
For e.g. Google on the other hand they got ONIX feeds set up no questions asked (ONIX is an XML format for feeding larger book catalogues to retailers) and pushed their entire ca. 200 book backlog right out of the gate. But Google e-book sales are tiny compared to Amazon - last I heard it was about 5% of their revenue.
The problem for Amazon - and for everyone else - is that for anyone but the biggest publishers Amazon is so dominant that most of the effort from people who want to push out high volume will continue to target Amazon.
Scams aren't even the worst problem for trust between readers and writers.
LLMs allow "writers" to generate prose faster than readers can read it. This is a fundamental break in the bargain between readers and writers: However long it takes someone to consume a piece of writing, it almost certainly took the author longer to write it. Even if the prose is crap or the premise awful, the author had to think it was worth sinking real time into making it.
My colleagues and I have implemented a little guideline, which is that you should spend more time writing something than you expect the audience to spend reading it. Anything less is disrespectful.
You can also run it through an llm to get a rephrased summary, and ask them to read it for you to check if it's what they meant before you read it.
very good point.
Pre-2023 writing is the low-background steel of culture.
Oh... No it isn't... Everyone and their aunt was writing books about the JS framework of the month before that... Slop before the AI slop existed.
I don't think the universal applicability of Sturgeon's Law necessarily leads to a contradiction here.
My incredibly, almost insultingly oversimplified model for what I, a lifelong consumer of text, feels is happening here goes like:
90% of culture is "bad". 10% is "good". Previously, we'd iterate on the 10% and (I remind you: oversimplified) discard the 90%, but `derive :: culture -> culture` still follows Sturgeon's Law, so at the next iteration, 90% is "bad" and 10% is good. But now what's happening is we're iterating on the whole sample, and a polished turd is still feculent. So in the next iteration, 90% + 90% of 10% of the culture is "bad". And so on.
Most content is slop, whether books or TV shows or movies or journalism.
That's why we have IMDB Top 250 lists, and books called "Classics". Curating collections and having taste has always been important, and is even more important now.
AI hasn't really changed that. There is good investigative AI-assisted writing today and there is also fully AI-slop fly-by-night websites made in 15 minutes with Claude.
Curation and the creation of canons requires enough people to have a shared context and be able to build community among themselves. This has been undermined by the shift from early-internet arts websites to social-media platform's algorithmic feeds where people are easily atomized into personal silos. And once atomized, they don't know there are alternatives to slop.
For-profit taste-making websites either got acquired and extinguished, or are in dire straits now, and in any event their influence over core audiences (especially young people with time for discovery) has waned to almost nothing. The non-profit blogosphere blogging about arts and literature largely grayed out and was never replaced; you can still set up a blog today, but you'll be talking to the void, you won't attract any real slice of your topic's fandom community like you once could. A putative scene hangs on at Substack, but if you look closely it's largely hustling and grifting.
All in all, curation just isn't there any more at a societal level in the same way it was in the twentieth or early twenty-first centuries.
I maintain that the AI era is forcing us to finally build good filters.
Part of the outrage is so called high literary critics have been fooled at book prizes.
Maybe a revolution like the blind taste testing of fine wines needs to happen in literature.
Do you think that's the kind of book the article is about?
The one thing that we can prove with reasonable ease is that the time it took to write was compatible with the author being a human, and that the writing signature is that of a human. You can still fake it, but it is much more annoying and it will take you at least as long as it would take a human.
I wrote Possibly Made by a Human just for fun to do this, after I failed the Turing test in a comment here.
Actually made by a human: https://possiblymadebyahuman.com/H1LfVy72BZ
Of course it makes for awkward editing! I've just realized that "the writing signature" does not mean much. I mean the histogram of delays between keystrokes.
We could just say "AI has destroyed trust" and it's accurate.
First hand experiences myself:
I can't commission an artist without questioning if they are using AI, which isn't acceptable in the projects I am commissioning for.
I can't trust that my coworkers are actually testing the crap they're generating before wasting my time with runbooks that aren't even validated. It was bad enough that people miss important steps with assumed knowledge, but now we have that amplified by these types of documents that are no being AI generated.
I can't trust that my code wont be sucked up by some money greedy company at speeds unheard of before and benefit someone with out any credit to me doing something novel or interesting.
Even worse, I can't trust the smart people to not be idiots either. How many engineers do you know that have completely outsourced all the thinking to the robot, while we know fully that it's closer to a slot machine than it is a thinking machine?
Maybe even worse than that is the companies thinking they can reduce 10% of their workforce and compensate with LLMs. It's absolutely laughable, because I witnessed it first hand. Company I worked at fired my recruiter and the guy that was going to be my boss before my start date. My onboarding was practically non-existent and nobody wants to train their newbies anymore. The fear of being axed from your job at the first thing that MIGHT make a profit is also high. To every company that retained me through their layoffs, joke is on them, people like me want to see stability and that's just fiscal irresponsibility they're signalling.
Just some thoughts to illustrate it's not just writing. Trust is destroyed in any creative and professional en devour at this point.
> I can't commission an artist without questioning if they are using AI, which isn't acceptable in the projects I am commissioning for.
We've always judged the output, so why stop now. AI is just a tool. The artist still owns the output.
We pretty much never only judged the output. An oil painting is valued because it's an oil painting among other things. If people know you have worked on something for a long time, it's valued more than something you just came up with this afternoon. A chair crafted by a woodworker, even if it's worse in some ways, is valued over an IKEA chair produced in a fraction of the time.
Generative models are like outsourcing to cheap mass production factory, not an advanced woodworking tool in your basement.
Both of those things are visible in the output. An Ikea chair isn't denigrated because we're such snobs, but because it's made of inferior materials. If Ikea started selling solid hardwood bookshelves tomorrow at the same price as their other stuff, I'd be driving right over there!
> Both of those things are visible in the output
Yes, they are different, but even with woodworker's one worse we still value it more for HOW it's made, not just what it is at this moment in time
Ikea does sell some hardwood furniture, though I don't know if any of their bookshelves are. But it's definitely within the realm of possibility.
No, because you can get indistinguishable hand painted oil clones of famous paintings from Vietnamese rep shops and they obviously sell for far less than the originals.
Barring AI generated art and prose assets is valid, if for example you don't want to have the AI generation marker on steam, or you object to the use of AI for various ethical reasons. Also there's apparently quite the market of motivated purchasers that want to signal their anti AI values by buying from projects that don't use AI.
> If Ikea started selling solid hardwood bookshelves tomorrow at the same price as their other stuff, I'd be driving right over there!
better go quick, they'll end up losing a lot of money selling more expensive furniture at a cheaper price...
Hmm, I don't know that time invested inherently equates to quality. I would place FAR more value on a comfortable functional mass-assembled chair than some wobbly piece of crap some guy spent two weeks cobbling together in his garage.
It's not just time, it's also effort and other non obvious things like supporting a real human being
> I would place FAR more value on a comfortable functional mass-assembled chair than some wobbly piece of crap some guy spent two weeks cobbling together in his garage
If it's wobbly then sure, but what if it was made by someone with a modicum of expertise to make it not wobbly.
then again, i don't mind a good wobble, it's a fidget opportunity and i like to fidget
Time and effort are not indicitive of quality.
I suppose we're talking about "craft" vs "art." Making chairs is a craft. There are objectively "right" ways to do it. You can still be artistic about it, so long as the craftsmanship is sound. That being said, I will take a well-crafted chair over one that is not every time over the "artistic" one.
With regards to writing, I can't just throw word salad together and call it a day. But my writing style is my own voice - I pretty much write the same way I speak. With AI... well, after you've read enough of it, you notice that it, too, has its own voice. Like, I can distinguish the writing style of Gemma4 real quick. It smells like ozone.
I don't necessarily mind, so long as its telling me something coherent and accurate. You could argue it lacks originality, and it certainly lacks agency: But it IS usually well-crafted. I'll take it. Won't stop me from reading, say, PKD novels. But miss me with the artistry of, say, Finnegan's Wake, or even Ulysses, I'm apparently way too dumb for those (yes, I've tried). PKD would crank out multiple novels every year, Joyce took like decade to write incomprehensible nonsense.
Define quality.
> We've always judged the output, so why stop now. AI is just a tool. The artist still owns the output.
But we didn't. We relied heavily on a mix of trust and maybe occasional sampling. Having to verify everything is inefficient and exhausting.
> We've always judged the output, so why stop now. AI is just a tool. The artist still owns the output.
Who are we? What output? Organic certification has a long history in agriculture. To get certified requires more than just judging the output. My process are regulated or judged, and have been, using criteria other than the blinded output.
Even if this were true, it doesn't address the other element of the problem, which is that flooding the zone with horrible crap makes it harder to find the good stuff, and forces you to waste a lot more of your own time filtering crap.
Yes, I can usually identify AI slop within a couple paragraphs, but it still wastes my time to encounter it, and if it's my entire feed, it's going to drown out the good stuff I'd actually want to read. Same for literature at a larger scale, to say nothing of the countless scandals where obvious AI-written crap wins various writing competitions.
Imagine a relative passed and you commission someone to draw a portrait of them to show at the funeral. The artist has good reviews and all their work looks impressive; their portfolio has hundreds of amazing photorealistic portraits. It's expensive, but nothing is too good for a passed loved one.
After weeks of waiting, you finally get the portrait ...aaaaand it looks nothing like them. There's weird artifacting all over it and it looks like a supermodel imposter with meat-moss hair. You try to contact the artist, but they've completely vanished: socials deleted, emails unreplied, just gone.
Oh well, they owned the output, right?
High trust environments are inherently more efficient and cheaper so the destruction of trust will act like a tax on all activity. But that'll probably make the GDP go up so it will be read as an economic improvement, just like wrestling with health insurance companies.
Even when they are honest about it there's a cost. I keep receiving status emails where the first line says "Generated with AI, be sure to check for accuracy". Apparently, it's more efficient to send an email with unverified information to twenty people and tell them to each individually check it than it is for the sender to just make sure it's accurate before they send it out. Maybe they're just asking their agent to do the check for them though.
"Maybe even worse than that is the companies thinking they can reduce 10% of their workforce and compensate with LLMs"
It's significantly dumber than that. Companies are paying to pour all of their domain and institutional knowledge, processes, and data into the training pipeline of tools their competitors use. Even better, they ignore the fact that as these tools gain competency the owners of these systems are guaranteed to start spinning up integrated industry verticals in direct competition.
Tried to tell that to my executive suite and they told me I was being ridiculous. Showed them OpenAIs "digital program management" for USG projects and it was like they suddenly went blind. Half a beat later it was like I never uttered a word and they were back to talking about "becoming a leader in AI." We're a fucking consulting company that can't even put small, hand written projects written by domain experts in front of the client because of our joke of an IT department. We're never going to credibly be "leaders in AI."
The charitable interpretation here is your executive suite all have exit strategies with < 2-5 year timelines. All alternative interpretations suggest profound incompetence.
Whoa! It's like a lightbulb turned on over my head. Can you say more about this? How do you see those larger patterns? I am missing social cues like this at my workplace and this is my first experience on a sinking ship of a business.
I posted this article yesterday in my company's book club channel Also yesterday I went out for some beers with a friend who has recently completed a science fiction trilogy, result of years of work, self-publishing it and selling a few thousand copies. It's pretty good stuff and definitely a life achievement! Anyway we discussed that article. He said that things got really bad in between his first book 6 years ago and third one just now, the amount of self-published slop that's denial-of-servicing everyone is just humongous. He said there are people publishing 50 books a year! 50! I mean, that's a below lifetime's output for a most really good writers! As he put in "5 dollars in compute to make a garbage book to publish on amazon to produce 30 bucks of revenue from unsuspecting half a dozen people is worth the squeeze so scammers/sloppers do it". That's why we can't have nice things.
People need to keep reading books, and write with their own pen and cursor to keep up their language skills, or it will be quite risky.
When reading takes more effort than writing, trust is inevitably a victim.
The Great Automatic Grammatizator
Things are ogre in the fanfiction and webserial scene. Everything written after 2024 REEKS of AI, and most of them, over 90% are obvious AI. It's very easy to spot if you're advanced reader, AI style has a lot of tells and idiosyncratic phrases (and please don't disclose them). There are meta tells as well: unrealistic output volume for a human writer, having a patreon for fanfiction, etc.
Are you kidding me? For a general audience:
The absolutely fraudulent blog post to nonfiction "bestseller" pipeline already crushed nonfiction writing over the last decade or more.
Good fiction has always been niche. The rules are the same, read the best.
Well, this was all foretold in 2019 when OpenAI held back GPT-2, an unusable model by today's standards, for fear of making misinformation and fraud too easy. I remember when the internet, once the preserve for businesses and universities, came to the home, and I said, wow, kids are going to watch a lot more porn. As with any life-changing technology, you take the good with the bad.
Its the reverse turing test: can a human convince another human (or bot screener) that they are human?
We need legislation banning ai from imitating/impersonating humans and creating confusion about whats real. And people using ai to make their art need to label it or else theyre just plagarizing
I mean, even ghost writers arent making the world a better place
> We need legislation banning ai from imitating/impersonating humans and creating confusion about whats real.
The obvious problem is that if there is an unambiguous way to tell that something is AI-generated then you don't need the law, and if there isn't then you can't enforce it.
>I mean, even ghost writers arent making the world a better place
I think people should be allowed to put their name on anything, be that hired work or AI generated. It’s only their own credibility and good name at stake.
Sure, but I disagree when you’re selling that thing.
In a large society trust is necessary because you can’t know everyone. So I think it’s reasonable that at least if you’re selling something you can’t lie about it, including whether you wrote it or not.
And yes, I’m aware ghost writing is a long established practice and I think it’s always been deceitful.
it seems like a slam dunk that we should own our likeness legally, but the scam lobby is very powerful at the moment
Is that a slam dunk? Why should someone have a right to prevent you from criticizing them by using their likeness? Should a billionaire be able to sue the newspaper for a satirical editorial cartoon?
AI has destroyed my trust in pretty much all content. It sucks. I used to enjoy watching video content online (YouTube, Instagram, whatever) but now, whenever I'm watching something, I'm always thinking, "Is this more fake-ass AI slop?" I spend more time trying to find a person with six fingers on one hand or whatever than I spend consuming the actual content. Same goes for new music. AI has ruined online content for me. Honestly, now that I think of it, I should probably be grateful. I waste a lot less time watching YouTube Shorts now. So yeah, thanks AI.
This is multi-faceted and I agree that AI is destroying some levels of trust, but many of the publications on the Internet were never deserving of trust anyway so I'd describe it as formalizing or recognizing distrust or lack of credibility and, in a manner of speaking now that the tide is going out we can see who has been swimming naked.
One of the other facets is the idea that trust is now being lost due to AI. It's unfortunate that it seems that publications like The Atlantic (and others) are now going to scapegoat the lost of trust that they've earned over the last decade or so to be about AI when it's actually about bad, biased writing with a default point of view, and lack of intellectual honesty and an inability to facilitate intellectual debate and discussion.
As some of these publications continue to spiral into irrelevancy I think what is a bit tragic is they'll never really understand why. They'll blame AI, which will likely destroy all but the top-tier sites, but in reality the quality of the writing and honesty went down the drain next to their ad-filled garbage pages and other spam.
I don't think there ever was generic trust between writers and readers. Trusting the written word just because it's written by someone or published by some newspaper was a huge mistake before AI was in the picture. The trust was exploited by many actors with agenda - politicians, journalists, academics and just plain grifters publishing another "self-help" book.
You could get a glimpse when a newspaper, even a very respected one published an article about your domain of expertise - I am sure I wasn't the only one who realized that if an incomplete, simplistic or just plain wrong text could be reviewed and published then probably a big part of other work is of similar quality - you are just not in position to judge it so you naturally trust it. Sadly the same goes for academic world - millions of papers gets published - a lot of them not worth the paper they are written on. You are just rarely in position to say.
What AI has done to it is actually a favor: it exposed what was always there but covered well enough to deceive big part of the population. Now we need to reevaluate. Come back to fundamental mechanisms. Can the claims be verified? Maybe in formal manner. If not who the author or someone authorizing the text is. What did they achieve in their life? What experiences they have lived through? What work have they done? What incentives do they have to publish it? If you can't answer those questions the text is irrelevant as a source of information. It turns out not everyone is worth listening to.
It was always the case. It's just easier to see now.
Internet already lost faith with paywall site like this.
And how could I trust if their article is not written by AIs at this point?
Trust in media corporations died a long time ago, AI has just accelerated the trend. When human ‘journalists’ write to a script handed to them by the media’s editors and owners (eg ‘write articles that convince our readers to maintain their confidence in the <dot-com, subprime, AI> market’ or ‘write articles that convince our readers that <regime change, tariffs> is needed’), and the result is a debacle, then trust is eroded and questions arise about the journalist’s integrity and/or intelligence. Then the owners and editors find new fresh journalists to generate new stories to ‘restore trust’.
With AI, the editors and owners can just send their scripts to a machine and it’ll write the narrative as requested - but without a human face on that article. However, most readers would not accept such material as trustworthy.
There is another reason that AI reduces trust in media, which is that AI now makes it trivial to trace media ownership up the tree to the consequential owners, look at their other economic interests, and examine if the articles they print tend to favor those economic interests. Applying this analytical framework to the Atlantic itself is an interesting exercise.
Q1: Would it be entirely unfair to characterize the Atlantic as ‘yet another mouthpiece of the neoliberal project instantiated in the United States in the 1970s’?
> Ever since, I’d worried that the misrepresentation might somehow have a real-life impact
Doubt is good for the brain & mind.
But either the author is AI or she--Rachel Kadish, born in '69--needed something AI related to write about.
There is no trust between writers & readers. People either distrust and scrutinize or they like or dislike how the reading went. Some don't even judge but just shrug it off whatever impact the reading might have had.
The problem is that the AI conversation is reinforcing cultural polarity, especially among experts and people who are at least well enough off, so any methods to ignore the actual problems, which have been ignored in the past by most people almost by convention, are welcome and this article seems, at least to me, to be the result of one of these applied methods.
> There is no trust between writers & readers. People either distrust and scrutinize or they like or dislike how the reading went. Some don't even judge but just shrug it off whatever impact the reading might have had.
I interpret this more as an underlying social contract: Both writing a book and reading it used to be laborious. Now they're not.
Reaching out to an author about specific details used to mean you've deeply engaged with the work. Writing a book used to mean you spent lots of time understanding and writing about the topic.
Neither is now true, so it makes it harder to distinguish what's worth paying attention to.
I've definitely seen this happen with myself. When I shop for books, I do extra research into the author and publisher for anything published after 2022.
Agreed I also think, what unique perspective am I getting if it's just AI? If I want to know what AI thinks I could ask it myself, I want more diverse opinions.
> what unique perspective am I getting if it's just AI?
It is entirely possible for existing AI tools to introduce you to a perspective on something that is neither new or unique to the world, but which IS new to you.
Whether what represents is a wholly unique perspective or not is kinda irrelevant, so long as it's unique to you.
For example - most popular science or history books aren't offering up wholly unique perspective that nobody in that field of study has ever thought about before / ever committed to paper. Most of them are providing an engaging overview of widely studied topics in a way that is accessible and engaging to the average enthusiast.
Sure, the more in-depth your knowledge of a subject is the harder it is for those books to give you a unique perspective...but that is true of human authors too (as anybody who has read multiple books on the same subject will surely attest).
Also - without getting all Rumsfeldian...there are things you don't know that you don't know...and whilst with the right approach you could get AI to enlighten you, there can be value in someone/something having already curated that content together for you.
Just to be clear here - i'm not arguing in favour of AI books or suggesting they are as good/equivalent to human-authored books. I don't believe they are, i'm mostly questioning the logic that because Ai isn't producing some unique perspective it's therefore of no value).
> It is entirely possible for existing AI tools to introduce you to a perspective on something that is neither new or unique to the world, but which IS new to you. Whether what represents is a wholly unique perspective or not is kinda irrelevant, so long as it's unique to you.
True, but then the but then the question becomes why you bought a book instead of just asking AI yourself and getting a personalized version.
Buying an AI-generated book just puts an intermediary between you and the AI.
Bingo. I read plenty of AI writing. My AI writes it for me. The way I like it. Usually along with obvious action items already drafted.
There may be a market for sharing useful prompt ideas. There isn’t one—broadly—for sharing the outputs.
> There isn’t one—broadly—for sharing the outputs.
And there probably won't be. If I want something made by AI, I can get something tailored to my exact tastes and without the delta that typically exists between a product on the market and the specifications I'd hand to someone to make something for me.
> There is no trust between writers & readers. People either distrust and scrutinize or they like or dislike how the reading went. Some don't even judge but just shrug it off whatever impact the reading might have had.
That suggests that writer reputation and the reader’s past experience are of no importance. But before you even get to read something, you have to decide to do so. On what basis? And how might a bad experience colour future decisions?
> born in '69
I can't see the article and I've stopped reading archive.is/.today etc., so can you explain what you mean there — are you just snarking at her age?
If you are coming for Gen X like that, I can tell you that you are going to have a rough time of it.