X Didn't Fix Grok's 'Undressing' Problem. It Just Makes People Pay for It
wired.comSaying cisgender is bad on Twitter but CSAM is not. Very weird.
Not remotely surprising, nor sadly is the fact that hacker news flag-killed this story.
It's not Hacker News. It's Musk fans on HN. The article is flagged (anyone can do it) but not dead. My reasonable comment elsewhere in this thread was also flagged by Musk fans but it's still alive.
> it’s not hacker news…
It’s not, but they could fix the issue by raising the flagging threshold for Musk-related posts.
As if it’s a non-profit organisation, not even remotely connected to him, Thiel et al.
It's hackernews because if it was Musk's fans the mods would unflag it
The mods absolutely endorse it though so in that sense it very much is them. They tend to be extremely dishonest and evasive when confronted directly about it but I mean anyone who has an account here can see with their own lying eyes that this happens multiple times a day, every day and it’s simply not plausible that it’s anything else other than something they support.
The purpose of a system is what it does. If the system did something different from its purpose, they would change it. I'm sure it's also intentional there's no vouch button for posts. This will change once every high quality post is flagged to death.
> there's no vouch button for posts.
There is. But seems like it’s only for [dead], not [flagged].
I suspect dead usually means shadow ban, at least for comments, and vouch is a way to selectively show through community support a high value comment from an otherwise abusive user. Where flagged is overt, already applies just to that one comment, and vouching in that case wouldn't really make logical sense. Unless we want people to be able to wage flag wars.
Maybe it’s time for a flag strike
Oh the bots got me. Out of nowhere too. This site is fucking rigged
Sad to see you downvoted for telling the simple truth.
But posts about the mass flagging also get deleted, so anyone who isn't checking the 'active' page has no idea about it all.
The gap between the rules as they are officially written down and the rules as they exist in practice is so wide that it’s basically unrecognisable when you start putting them side by side.
The mods will just gaslight you as well about the entire thing. I know for some unknown reason it’s considered bad form to have anything other than a high opinion of the job they do but I think they do a bad job. I’m not saying it’s not a hard job or anything, I just think they are actively bad at it.
I am no Musk fan but this "outrage" is entirely dishonest and stupid. If you have a problem with the images Grok generates, then don't use it. This is bottom of the barrel journalism and it is no wonder the story is flagged.
So you support undressing little girls I take it?
I don't really know what you take.
My americanism didn't land. I mean I understand you support pedophilia as long as it's cartoon or AI generated form.
No answer, yet they commented in another thread. This tells me (and you) what we needed to know.
Why should I answer, you made up your judgement from my initial comment and there was no substance in the reply. My mistake was to reply in the first place. For the record, I understood the colloquialism even if English isn't my native tongue.
So CSAM is fine, as long as those who don't like it choose not to use the platform.
Sorry, then you're a pedo-excusing ejut.
It's not about judgement, or mistakenly replying, it's about what you've written. If you didn't mean that, then own it and clarify. Because that's what you said.
Makes complete sense when you view it through the lens of Musk's opinions.
Even given Musk's opinions, he clearly understands that the general public doesn't want kids getting hurt, he demonstrated this by saying Trump was in the Epstein files and by repeatedly saying the UK government isn't doing enough to stop child abuse and opining about a UK civil war.
His hypocrisy, his position on the main-character-syndrome-to-narcissism spectrum, him getting a kick out of trolling everyone, or him having straight up psychopathy: whatever it is, I find I no longer care.
> he clearly understands that the general public doesn't want kids getting hurt
This may be giving him too much credit, the only thing we actually know is he thinks being accused of being a pedophile is bad. We know this because he's done it to several people, and flips his shit when it happens to him or his platform. He doesn't actually seem to care about pedophiles or pedophilia given his on going relationships with people he's accused.
Mm. Took me a moment to see your point there, but I think you're right.
If he's only operating on the impact of the words, and ignoring the existence of an observable testable shared reality behind the words, then yes, accusations (either direction) are more damaging in his mind than being seen to support or oppose whatever.
Which is, ironically, a reason to *oppose* absolute freedom of speech, when words have power beyond their connection to reality the justifications fall short. But like I said, I don't care if his inconsistency is simple hypocrisy or something more complex, not any more.
> If he's only operating on the impact of the words..
> Which is, ironically, a reason to oppose absolute freedom of speech...
Since the former theory of mind can't explain the latter behavior, I guess it's wrong then, right?
Please elaborate, especially note that people on the internet loudly disagree if Musk's behaviour is supporting or suppressing freedom of expression and I have no way to guess what your position is without spending a lot of time diving into your comment history (a superficial glance didn't disambiguate).
You went along with the theory of the poster and then presented evidence against it as ironic rather than evidence against.
"I have a theory that Musk always wears green because he loves money. It's ironic that he wore red that one time."
Y'see, this is why I asked you note that people disagree if Musk's behaviour is supporting or suppressing freedom of expression.
First: Musk says he is a "free speech absolutist", his behaviour is incongruent with what he says, he does in fact ban people (and file lawsuits, and fire people) for expression of things not forbidden by law.
Second: Your second quotation of my words, these words:
> Which is, ironically, a reason to oppose absolute freedom of speech...
With these words I am not saying anything about if Musk does or does not support free speech (that's an independent statement which I also believe yields "does not"), I'm saying that the existence of people who operate on the basis of words having power beyond their connection to reality is a reason for any random person or group of people to have limits on free speech, and that my reasons why are summarised by the words after the end of your quotation: the justifications (for absolute freedom of expression) fall short (when such people operate).
Has Musk developed a more nuanced understanding of freedom of expression since claiming to be a "free speech absolutist"? If he has, he's evil rather than simply a hypocrite. But I don't care which.
Fits the pattern. One of the two F-words will get your post flagged faster than the other.
Full Self Driving?
chef's kiss
Concerning.
Concerning.
It also made it harder to track. One way to see what Grok is doing is to look at the Grok account’s replies. So you can see the image it generates in response to someone - for example - undressing a woman who appears in a photo. You can then go visit THAT thread to see what the exchange with Grok was, which often would show a long series of lewd images. A few days ago, nearly the ENTIRE stream of the Grok account’s replies at any moment were deepfakes without consent. Mostly in response to women’s posts, but sometimes to generate racist attacks.
I’m not against people using AI to generate a fantasy image for their own needs. I guess in a way it’s like what people imagine in their own heads anyways. But I do think it is problematic when you share those publicly because it can damage others’ reputation, and because it makes social media hostile to some groups of people who are targeted with misogynist or racist deepfakes. It may seem like a small problem but the actual final effect is that the digital public square becomes a space only for identity groups that aren’t harassed.
Well, grok just automates what you can do by hand if you want, and there's not much to stop me just drawing out these same types of images manually if I want.
The problem is that doing this would get me banned. Shouldn't using Grok in this way get you banned similarly?
Automation makes it easy for everyone to do it, on demand.
That's fundamentally different to "You can make this thing if you're fairly skilled and - for some kinds of images - have specialist tools."
Yes, you should be banned for undressing people without consent and posting it on a busy social media site.
Why would I need to be skilled? Isn't the issue the content not the quality?
The quality is absolutely part of the issue. Imagine the difference between a nude stick figure labeled your mom, and a photorealistic, explicit deepfake of your mom.
Do you find the two equally objectionable?
Well also in context the stick figure could still constitute sexual harassment.
If a big boobed stick figure with a label saying "<coworker name>" was being posted on your social media a lot such that people could clearly interpret who you were talking about, there would be a case for harassment but also you'd probably just get fired anyway.
Yes, but in that case everyone would understand the image is a crude depiction of someone—judging the poster—and not a real photograph—judging and embarrasing the target.
Well, if we just guarantee that we put "AI Generated" at the bottom of those images, it will be clear it's not a real photograph, and then this problem disappears?
It’s impossible to guarantee that. As soon as you add that message, someone will build a solution to remove the message. That’s exactly what happened with OpenAi’s Sora.
You've avoiding the question. Assume there is a technical solution that makes these generates images always obvious as generated.
Where is the actual problem?
Is it that it's realistic? Or that the behavior of the person creating it is harassing?
This is pretty straight forward.
It is pretty straightforward: the problem is both.
Automation makes it easy for everyone to do it, on demand.
That's fundamentally different to "You can make this thing if you're fairly skilled and - for some kinds of images - have specialist tools."
Yes, you should be banned for undressing adults and kids without consent and posting it on a busy social media site.
Why? The people creating and operating the CSAM/revenge porn/depfakes creation and distribution platform are the ones who are culpable. The users who are creating text prompts are just writing words.
There’s a frantic effort to claim 230 protection, but this doesn’t protect you from the consequences of posting content all by yourself on the site you own and control.
> the CSAM/revenge porn/depfakes creation and distribution platform
Which, in this case, is Twitter itself, no?
> The users who are creating text prompts are just writing words.
With highly specific intentions. It's not as if grok is curing cancer. Perhaps it's worth throwing away this minor distinction and considering the problem holistically.
> With highly specific intentions
Intentions to pull the CSAM out of the server full of CSAM that twitter is running.
Yes, you are making the flailing argument that the operators of the CSAM site desperately want to establish as the false but dominant narrative.
If you have a database full of CSAM, and investigators write queries with specific intentions, and results show that there is CSAM in your database: you have a database full of CSAM. Now substitute 'model' for 'database.'
Grok enables their behavior.
An investigator does not _create novel child porn_ in doing a query.
You're making a fallacious argument.
> An investigator does not _create novel child porn_ in doing a query.
And a prompt, without being aided and abetted by twitter, doesn't "create novel child porn" either. A prompt is essentially searching the space, and in the model operated by twitter it's yielding CSAM which is then being distributed to the world.
If twitter were operating in good faith, even if this was the fault of its customers, it would shut the CSAM generator operation down until it could get a handle on the rampant criminal activity on its platform.
How is this not equivalent to saying that gun manufacturers should be held responsible for gun related murders?
Because the gun manufacturers aren't doing anything illegal, whereas running a service which creates CSAM is, and so is distributing CSAM.
The tortured equivalent analogy would be if gun manufacturers ran a service where they themselves would murder somebody with their guns whenever someone posted a tweet with the victim's name
If you can post "Smith and Wesson, please kill this person" and then they die, someone from S&W should go to jail, yes
Obviously anybody can post gross things by running an image generation/editing tool locally and publishing the results. People then mostly blame the poster whose name it then appears under.
Seems like a pointless and foolish product design error for X/grok to publish arbitrary image generation results under its own name. How could you expect that to go anything but poorly?
It's not just a matter of publishing it under its own name. It also massively reduced the friction to do so compared to needing to run the image through an external tool and upload the result. That friction would greatly reduce the number of people who do it.
In the US, it used to be that if you made credible threats against people you could/would be prosecuted. Social media made it so common in that no district attorney goes to the trouble of actually finding prosecuting people for doing this.
We can expect the same level of institutional breakdown with regards to various types of harassment, misappropriation, libel, and even manufactured revenge porn from AI.
Now the credible threats are being made by Trump fans, and those are protected from criminal prosecution by his long shadow.
It’s even worse as the requestor doesn’t vet and approve the image. That seems to have removed editorial control from the requestor. This bot could also mess with user who are not trying to do bad thing X, but the black box bot decides to throw in some offputting stuff and then also associate your name with it.
I keep coming to the same conclusion with X as they did in the 80’s masterpiece War Games.
The through line of the success many AI ventures seems to be the shifting of liability, sometimes towards the consumer, sometimes to a third party, and sometimes to the nebulous void.
With self-driving cars, companies appear to be offloading the liability of their own software to the driver (“it’s not _fully_ self driving, hands on the wheel!”), or even pedestrians (“wasn’t in the crosswalk”).
This case appears no different. The thought experiment would be, for me at least, is if Grok was just any other user that was somehow capable of creating these images at scale? Who would be liable, the image prompter or the image generator?
Or how about a system that asked the user to approve or disapprove of the image being posted before it’s posted?
Or how about a system that posted the generated image under the prompters username, and not Grok’s?
Is there any way to provide a service where an image manipulation bot is mentioned in social media replies and it doesn't lead to total chaos?
From what I saw the 'undressing' problem was the tip of the iceberg of crazy things people have asked Grok to do.
> Is there any way to provide a service where an image manipulation bot is mentioned in social media replies and it doesn't lead to total chaos?
It may be a failure of imagination on my part, but I can't imagine a bot limited to style transfer or replacing faces with corresponding emoji would cause total chaos.
Even if someone used that kind of thing with a picture from an open-casket funeral, it would get tuts rather than chaos.
> From what I saw the 'undressing' problem was the tip of the iceberg of crazy things people have asked Grok to do.
Indeed. I mean, how out of touch does one have to be to look at Twitter and think "yes, this place will benefit from photorealistic image editing driven purely by freeform natural language, nothing could go wrong"?
This very clearly violates both Apple's App store and Google's Play store rules.
Why is X still on the app stores?
Special treatment for big players.
I have seen full blown porn on instagram too. Ads. Porn ads. They look exactly like porn ads on porn websites.
I've seen it on Insta as well, but I think the authors use some very clever processing to hide it from the detection algos, and it quickly gets reported and taken down.
I've seen porn in Google's own Chrome too.
Most CSE issues are in the UK are on Snapchat, followed by Instagram, a few others then X - I’m imagining Apple doesn’t delete these apps because it’s smarter to target the abusers.
"Dear Email Recipient, I am a South African Billionaire with friends in powerful places..."
I agree with the sentiment, but nobody even needs to make these sort of threats or asks anymore.
It is all a well-defined implicit caste hierarchy at this point and anyone with enough net worth and a willingness to publicly fellate the orange dong gets a protected spot on the 2nd tier of the pyramid.
Sarah Chayes studies corruption: https://www.newyorker.com/magazine/2015/01/19/corruption-rev...
In her book she writes how in Egypt, to get a job as police, you need to bribe your boss and pay a monthly tribute. Being a cop was a chance to join the Mafia squeezing the public for bribes. Your boss pays his boss, and so on to the top. So when the public protested against the regime, the police were especially brutal against them, because all the cops were in on the scam and wanted to preserve it/if the regime collapses their necks are also on the line.
Looking forward to the ICE thugs beating and shooting Americans who want to bring down the current lawless regime... Or is that already happening?
Trump hires fuckwits like Noem or Hagseth who (probably) knows they're dumb fucks who would never otherwise be in their positions, and they would suck any dick, metaphorical or even perhaps real mushroom-sized ones, to remain in power.
Another post about X's child porn bot flagged as soon as it got any traction. See also:
https://news.ycombinator.com/item?id=46548451 https://news.ycombinator.com/item?id=46503199
There is a real reason though. Limiting it to verified users is the easiest way to have a KYC on everyone generating images. That way they can respond to legal requests with the KYC of the account that asked grok to undress a minor.
The constant comparisons with Photoshop are so disingenuous. We all know what the difference is.
If Adobe had a service where you could E-mail them "Please generate and post CSAM for me" and in response, their backend service did it and posted it, that's a totally different story then the user doing it themself in Photoshop. Come on. We all know about tech products here, and we can all make this distinction.
Grok's interface is not "draw this pixel here, draw this pixel there." It's "Draw this child without clothing." Or "Draw this child in a bikini." Totally different.
And the service was designed by Grok, hosted by Grok, you interact with it through systems controlled by Grok, at a surface level Grok makes decisions and grok makes the output. And it is quite possible the Grok knew that illegal image creation was possible. 99.9% of the work to make those images is within grok.
Not to be too pedantic, but I think you mean Grok with a k. Groq with a q is a separate AI hardware company.
Thanks, changed that.
I see at least 2 axes here: * Should access to a tool be restricted of it is used for malice * Is a company complicit if its automated service is being used for malice
For 1, crowbars are generally available but knives and guns are heavily regulated in the vast majority of the world, even though both are used for murder as well as legitimate applications.
For 2, things get even more complicated. Eg if my router is hacked and participates in a botnet I am generally not liable, but if I rent out my house and the tenant turns it into a weed farm i am liable.
Liability is placed where it minimises perceived societal cost. Emphasis on perceived.
What is worse for society, limiting information access to millions of people or allowing csam, harrassment and shaming?
It is not clear that limiting Grok limits information access to millions of people, so I think your premise is flawed.
There are plenty of other resources that could serve the same people that Grok serves. Further, the fact that we aren't having discussions about ChatGPT or Claude as CSAM generators also suggests that Grok could be limited in ways that it isn't being limited currently.
how is it different? i don't get it.
It's the frictionless aspect of it. It requires basically no user effort to do some serious harassment. I would say there's some spectrum of effort that impacts who is liable along with a cost/benefit analysis of some safe guards. If users were required to give paragraph long jailbreaks to achieve this and xAI had implemented ML filters, then I think there could be a more reasonable case that xAI wasn't being completely negligent here. Instead, it looks like almost no effort was put into restricting Grok from doing something ridiculous. The cost here is restricting AI image generation which isn't necessarily that much of a burden on society.
It is difficult to put similar safeguards into Photoshop and the difficulty of doing the same in Photoshop is much higher.
i think you have a point but consider this hypothetical situation.
you are in 1500's before the printing press was invented. surely the printing press can also reduce the friction to distribute unethical stuff like CP.
what is the appropriate thing to do here to ensure justice? penalise the authors? penalise the distributors? penalise the factory? penalise the technology itself?
Photocopiers are mandated by law to refuse copying currency. Would you say that's a restriction of your free speech or too burdensome on the technology itself?
If curl is used by hackers in illegal activity, culpability falls on the hackers, not the maintainers of curl.
If I ask the maintainers of curl to hack something and they do it, then they are culpable (and possibly me as well).
Using Photoshop to do something doesn’t make Adobe complicit because Adobe isn’t involved in what you’re using Photoshop for. I suppose they could involve themselves, if you’d prefer that.
so why is the culpability on grok?
Because Grok posts child porn, which is illegal. Section 230 doesn't apply, since the child porn is clearly posted by Grok.
where did grok post child porn?
X, formerly Twitter.
You could drive your car erratically and cause accidents, and it would be your fault. The fact that Honda or whoever made your car is irrelevant. Clearly you as the driver are solely responsible for your negligence in this case.
On the other hand, if you bought a car that had a “Mad Max” self driving mode that drives erratically and causes accidents, yes, you are still responsible as the driver for putting your car into “Mad Max” mode. But the manufacturer of the car is also responsible for negligence in creating this dangerous mode that need not exist.
There is a meaningful distinction between a tool that can be used for illegal purposes and a tool that is created specifically to enable or encourage illegal purposes.
You don’t understand the difference between typing “draw a giraffe in a tuxedo in the style of MC Escher” into a text box and getting an image in a few seconds, versus the skill and time necessary to do it in an image manipulation program?
You don’t understand how scale and accessibility matter? That having easy cheap access to something makes it so there is more of it?
You don’t understand that because any talentless hack can generate child and revenge porn on a whim, they will do it instead of having time to cool off and think about their actions?
yes but the onus is on the person calling grok and not grok.
why do you think that?
So, is it that you don’t understand how the two differ (which is what you originally claimed), or that you disagree about who is responsible (which the person you replied to hasn’t specified)?
You made one specific question, but then responded with something unrelated to the three people (so far) who have replied.
Wasn't this entirely predictable and inevitable? The genie is out of the bottle.
Where can we realistically draw the line? Preventing distribution of this sort of shit is impossible, anyone can run their own generator. CSAM is already banned pretty much everywhere, and making money off it certainly is, but somehow Musk is getting away with distributing it at a massive scale. Is it because it's fake? And can we even tell whether it's still fake? Do we ban profiting from fake porn? Do we ban computing? Do we ban unregulated access to generative AI?
X/Grok is an attractive obvious target because it's so heinous and widespread, but putting the axe on them won't make much of a difference.
How about we start at "not enabling users to directly generate nonconsensual porn of other users using your platform and then posting it as a reply to their content"?
Does any platform prevent that? HN doesn’t.
HN will ban accounts that do that.
Grok hasn't been banned from Twitter.
Would you (and others) be happier if when someone asked Grok to post something, it came from the person's own account?
If so we agree.
I said generating it using the platform. These creeps aren't just freelancing -- the images are being created by Twitter's servers via their native UI and published directly by their official @Grok account. They are utterly complicit in a way they wouldn't be if they were "just" permitting people to post AI revenge porn they created independently.
>CSAM is already banned pretty much everywhere, and making money off it certainly is, but somehow Musk is getting away with distributing it at a massive scale. Is it because it's fake?
It's because law is slow and right now the US government is completely stalled out in terms of performing its job (thanks in part to Musk himself). Things will eventually catch up but it's simply the wild west for the next few years.
Is government intervention even necessary? IANAL, and I don't know shit about US law, but if this this crap on X is illegal, then surely the courts can handle this and ban/fine/jail the responsible parties?
If this isn't illegal, then sure, government intervention will be required, laws will have to be amended, etc. Until that happens, what are realistic options? Shaming the perps? A bit of hacktivism?
The argument is X themselves are responsible, but they make it extremely easy and do the service for you. It's different.
Like, if I sell a gun and you go and shoot someone I'm not necessarily responsible. Okay, makes sense.
But if I run a shooting range and I give you zero training and don't even bother to put up walls, and someone gets shot, then I probably am responsible.
That might mean something like Grok cannot realistically run at scale. I say good riddance and who cares.
> Is government intervention even necessary? IANAL, and I don't know shit about US law, but if this this crap on X is illegal, then surely the courts can handle this and ban/fine/jail the responsible parties?
Government includes courts in American. And prosecutors are part of the executive branch in the US.
The federal courts in the US are theoretically non-partisian. But 2025 has shown that the Department of Justice has functioned as Trump's personal legal counsel. That's even happening as we speak with the kerfuffle behind the Central Bank (another non-partisan organization that Trump is desperately trying to make partisan).
As is, Musk probably isn't going to get confronted by this current DoJ. The state courts may try to take this up, but it has less reach than the federal courts. Other country's courts may take action and even ban X.
>what are realistic options? Shaming the perps? A bit of hacktivism?
Those can happen. I don't know how much it moves the needle, but those will be inevitable reactions. The only way out for the American people would be to mass boycott X over this, but our political activism has been fairly weak. Especially for software.
What AI can generate, AI can detect. It is well within the power of the social media companies to deal with this stuff. It’s not crazy to hope that hitting X has a meaningful effect not only on them, but also the others.
Musk is a business genius. He found the only way to make money with GenAI: Porn!
What happens if you ask it to undress Elon Musk? Not saying someone with a Xhitter account to burn should do this, but not not saying they should do it.
I'm only on Twitter once every 2 months but last time I checked the place was absolutely overflowing with images of Musk in a bikini.
Elon already did it to himself on his account
Oh well, hey… at least he did that.
Or to give Kier Starmer a Borat-esque outfit?
Musk and Trump have clearly talked about this idea that any attention is good attention because they both have started doing it to an extreme in the last year. Its like it doesn't matter how depraved or idiotic an idea is, the more outrage it causes, the better it is. To them, it's the only metric that matters at all.
Another question that should be asked is what culturally drives people to want to create lewd content of children and what should we change so that it stops happening? Obviously platforms should have safeguard against child porn and misogynistic defamation, but as a society we also need cultural changes so that people don't become pedophiles and sexists and that the ones that do get help with their glaring issues.
It's like saying "what draws people to murder"? At some level, on the scale of billinos of humnans, there are simply going to be morally corrupt people. Be it from clinical sickness or local conditioning. We can't "save" every person in this regard.
Society can create disincentives, but not cures.
While I agree with you, this doesn’t mean that the number of murderers cannot be decreased through societal changes
Outside of ending qualified immunity, we pretty much have the strongest disincentives for murder out there.
The only way from there I'd to have much better mental facilities to identify and treat (but not shame) aspects of sociopathy or psychopathy. But we're still very far from that reality.
Well I'd say one obvious solution would be to punish a company which deploys a tool to enable turn key sexual harassment at scale.
Might send a good message about consent ya know?
I think that's a naïve idea if you think punishing a company will have any affect on the situation.
Punish Musk; charge him with one count of producing CSAM per generated CSAM image. Charge him with one count of sexual harassment per nonconsensually generated and shared image.
Do this in state court, so it is not pardonable.
He is hosting a rape room.
He has the power to stop it, but does not.
It will tangibly lead to less CSAM on the internet, so yeah it will have an affect.
Obviously we can't just - poof - make people not child molesters or not murderers. But that doesn't mean we should sit on our asses and do nothing.
A very large portion of the harassment via these images is very very obviously motivated by humiliating people.
The person telling grok to comment on a thread by a woman with an image of her with her clothes off, on all fours, and covered in what appears to be semen is to hurt her. It is an act of domination. She can either leave the platform or be forced to endure a process that repeatedly makes her into a literal sex object as she uses the platform. Discussing something related to your professional work? Doesn't matter. There's now an image in the thread of this shit.
This is rape culture. There is no other word for it.
Gender theorists have been studying this very question for decades. But you'll regularly find this community shitting on that entire field of study even though I'm not sure if it is has ever been more relevant than it is today.
Take my upvote and thank you!
Why do you think it is culture specifically?
I don't think we're ever going to be able to eliminate various sorts of harmful desires and urges.
The same old misleading headline yet again. Grok continues to be free, whatever that X message said, it had so far no impact on Grok web and mobile app. They just switched off the part of their integration with X that let people generate images.
You can of course pay for Grok if you like, but that just buys you bigger quota (up to 50 videos a day is free), not new capabilities or less censorship.
For anyone unaware/uninformed, take a look at https://www.reddit.com/r/grok
Warning: it's quite gross
Worth noting that, in the UK, if there's CSAM on that page, you run a strong risk of prosecution[0]
[0] https://www.cps.gov.uk/prosecution-guidance/indecent-and-pro... ("downloading an image from a website onto a computer screen: R v Smith; R v Jayson [2003] 1 Cr. App. R. 13")
How long can we keep trying to put a finger in this particular dike? In five years, most people will be able to run a local LLM capable of whatever nefarious purposes they choose.
That's their business.
If there's a business operating for profit (and Twitter is, ostensibly) and their tool posts pictures of me undressed, then I am going to have a problem with it.
And I'm just some dude. It probably means a lot more for women who are celebrities.
"It's inevitable" isn't an excuse for bad corporate or personal behavior involving technology. Taken to its logical conclusion, we're all going to die, so it's just executing on the inevitable when someone is murdered.
The only excuses for bad corporate behavior are bad corporate laws and weak enforcement.
How do you get proper laws passed when the politicians are bought and paid for by the same corporations?
In the USA ... a company can declare bankruptcy and shed the debt / liabilities while a person cannot shed most debt after declaring bankruptcy. [0] [1] USA politicians favor companies over the people.
I personal support new corporate laws similar to California's three strikes law. Instead of allow companies to budget for fines the CEO and Executives go to jail with the corporation being broken up after habitually breaking same the laws.
[0] https://hls.harvard.edu/today/expert-explains-how-companies-...
That's often because of regulatory capture.
Is it not illegal defamation to have gen-AI post a deepfake in public? Photoshop existed before. Is it not illegal to post CSAM, regardless of where it comes from?
No other company would touch this sort of thing - they’d be unable to make any money, their payment providers would ban them, their banks would run away.
> Is it not illegal to post CSAM, regardless of where it comes from?
This is a great example of why "CSAM" is a terrible term and why CP was/is better. If you generate pornographic images of children using an AI tool it is by definition not CSAM, as no children were sexually assaulted. But it is still CP.
Fine, call it what you want, but CP of or appearing to be of real children is illegal. There’s some grey area for drawn stuff it seems, but at a certain point there IS a line.
Also what changed, over the past 20 years even hosting stuff like this, or even any pornography whatsoever, would get you pulled from every App Store, shut down by any payment providers. Now it’s just totally fine? To me that’s a massive change entirely decided by Elon Musk.
> no children were sexually assaulted
Generating pictures of a real child naked is assault. Imagine finding child photos of yourself online naked being passed around. Its extremely unpleasant and its assault.
If you're arguing that generating a "fake child" is somehow significantly different and that you want to split hairs over the CSAM/CP term in that specific case. Its not a great take to be honest, people understand CSAM, actually verifying if its a "real" child or not, is not really relevant.
>actually verifying if its a "real" child or not, is not really relevant.
It's entirely relevant. Is the law protecting victims or banning depictions?
If you try to do the latter, you'll run head first into the decades long debate that is the obscenity test in the US. The former, meanwhile, is made as a way to make sure people aren't hurt. It's not too dissimilar to freedom of speech vs slander.
> Is the law protecting victims or banning depictions?
Both. When there's plausible deniability, it slows down all investigations.
> If you try to do the latter, you'll run head first into the decades long debate that is the obscenity test in the US. The former, meanwhile, is made as a way to make sure people aren't hurt. It's not too dissimilar to freedom of speech vs slander.
There's a world outside the US, a world of various nations which don't care about US legal rulings, and which are various degrees of willing-to-happy to ban US services.
>There's a world outside the US
Cool, I'm all for everyone else banning X. But sadly it's a US company subject to US laws.
I'm just explaining why anyone in the US who would take legal action may have trouble without making the above distinction
Definitely a core weakness of the Constitution. One that assumed a lot of good faith in its people.
It, the difference between calling child pornographic content cp vs CSAM, is splitting hairs. Call it CSAM its the modern term. Don't try to create a divide on terminology due to an edge case on some legal code interpretations. It doesn't really help in my opinion and is not a worthwhile argument. I understand where you are coming from on a technicality. But the current definition does "fit" well enough. So why make it an issue. As an example consider the following theoretical case:
a lawyer and judge are discussing a case and using the terminology CSAM in the case and needs to argue between the legality or issue between the child being real or not. What help is it in this situation to use CP vs CSAM in that moment. I dont really think it changes things at all. In both cases the lawyer and judge would need to still clarify for everyone that "presumably" the person is not real. So an acronym change on this point to me is still not a great take. Its regressive, not progressive.
>It, the difference between calling child pornographic content cp vs CSAM, is splitting hairs.
Yes, and it's a lawyer's job to split hairs. Up thread was talking about legal action so being able distinguish the term changes how you'd attack the issue.
> What help is it in this situation to use CP vs CSAM in that moment. I dont really think it changes things at all.
I just explaied it.
You're free to have your own colloquial opinion on the matter. But if you want to discuss law you need to understand the history on the topic. Especially one as controversial as this. These are probably all tired talking points from before we were born, so while it's novel and insignificant to us, it's language that has made or broken cases in the past. Cases that will be used for precedent.
>So an acronym change on this point to me is still not a great take. Its regressive, not progressive.
I don't really care about the acronym. I'm not a lawyer. A duck is a duck to me.
I'm just explaining why in this legal context the wording does matter. Maybe it shouldn't, but that's not my call.
It's also irrelevant to some extent: manipulating someone's likeness without their consent is also antisocial, in many jurisdictions illegal, and doing so in a sexualized way making it even more illegal.
The children aspect just makes a bad thing even worse and seems to thankfully get some (though enough IMO) people to realize it.
I guess the difference is that one can point to the ToS and say "look we said no deepfakes" and block you if you upload a deepfake produced locally, but not if you use the built-in deepfake generator.
And it’ll still quite rightly be illegal.
So, because someone could hypothetically abuse their own child, we should stop trying to thwart child trafficking? Is that your line of argumentation, because if not, I don't understand what you are saying.
I am saying that half-assed measures that rely on the imaginary goodwill of megacorps is getting us nowhere, fast.
They don't need to have good will, they just need to not do the bad thing.
Explain why any for-profit enterprise would ever take any action that wasn't in it's own interest, unless compelled by law.
Big turn around from needing good will to something being in their interest
I fail to see what "good will" has to do with anything.
Then why did you mention it?
>"I am saying that half-assed measures that rely on the imaginary *GOOD WILL* of megacorps is getting us nowhere, fast."
Obviously I think that AI generated undressing pictures of people, especially minors, is bad and there should be safeguards against that. But how is it different from other tools like doing it manually with photoshop? Also it has been shown that many other tools like ChatGPT and Gemini/Nanobanana can also do it with sufficiently creative prompting.
I also did scroll through the public grok feed and the AI generated bikini pics were mostly Onlyfans creators requesting their own fans to generate these pictures (or sometimes generating them themselves).
You know the answer to this but I'll just say it: Its different in that it requires no skill and can be done by anyone instantaneously at scale.
You know this but somehow are rationalizing this game changing fact away.
Yes, people can draw and photoshop things. But it takes time, skill, dedication, etc. This time cost is load bearing in the way society needs to deal with the tools it has for the same reason at the extreme that kitchen knives have different regulations than nuclear weapons.
It is also trivially easy for grok to censor this usage for the vast majority of offenders by using the same LLM technology they already have to classify content created by their own tools. Yes, it could get jailbroken but that requires skill, time, dedication, etc; And it can be rapidly patched, greatly mitigating the scale of abuse.
> But how is it different from other tools like doing it manually with photoshop?
The scale of effect and barrier to entry. Both are orders of magnitude easier and faster. It would take hours of patience and work to mostly create one convincing fake using photoshop, once you had spent the time and money to learn the tool and acquire it. This creates a natural large moat to the creation process. With Groom it takes a minute at most with no effort or energy needed.
And then there is the ease of distribution to a wide audience, X/Groom handles that for you by automatically giving you an audience of millions.
It’s like with guns. Why prevent selling weapons to violent offenders when they could just build their own guns from high quality steel, a precision drill, and a good CNC machine? Scale and barrier to entry are real blockers for a problem to mostly solve itself. And sometimes a 99% solution is better than no solution.
This thing with guns was a legitimate argument for banning or regulating 3D printers a few years ago though and I'm glad that we didn't end up with restrictions on that front. With affordable desktop CNC machines capable of making metal parts coming soon, I hope those won't be subject to too many restrictions also.
> Obviously I think that AI generated undressing pictures of people, especially minors, is bad and there should be safeguards against that.
It's not obvious to me that this is your position. What safeguards do you propose as an alternative to those discussed in the article?
I am for moderation and strong penalties for users that use it in that manner. Anyone who uses grok to generate an undressing image of someone without their consent within 5 seconds should probably go to jail or whatever the penalty is for someone spending 5 hours to create revenge porn with photoshop.
But I'm not sure if the tool itself should be banned, as some people seem to be suggesting. There are content creators on the platform that do use NSFW image generation capabilities in a consensual and legitimate fashion.
Photoshop is a productivity tool, and the pricing supports that assertion.
Grom is much much less censored on purpose. I work in image editing and outside of very few people, hardly anyone uses Grok for professional work. Nano Banana Pro is used for the most part.
But for NSFW work it dominates. It’s clearly deliberate.
If you generate CSAM, whether using LLMs, photoshop, or any other tool, you are breaking the law. This would apply if you could somehow run Grok locally.
When you use a service like Grok now, the service is the one using the tool (Grok model) to generate it, and thus the service is producing CSAM. This would also apply if you paid someone to use Photoshop to produce CSAM: they would be breaking the law in doing so.
This is setting aside the issue of twitter actually distributing the CSAM.
The Photoshop equivalent would be "an Adobe artist does the photoshop for you and then somehow emails it directly to your target and everyone who follows them."
Drawing indecent photos of children with Photoshop is also illegal in lots of countries and any company creating them for profit would be liable.
Exactly. And the fact that companies do it with impunity is another hint that we're living in late stage capitalism.
If an individual invented a tool that can generate such pictures, he'd be arrested immediately. A company does it, it's just a woopsie. And most people don't find this strange.
I think intent probably matters and that this gets into the "you know it when you see it" definition realm where we debate the balance between freedom of speech and security of person. ie. just how easy Photoshop, a VCR, a DVD burner app, etc. makes it for you to crime and how much are they handholding you towards criming?
I think this is an important question to ask despite the subject matter because the subject matter makes it easy for authorities to scream, "think of the children you degenerate!" while they take away your freedoms.
I think Musk is happy to pander to and profit from degeneracy, especially by screaming, "it's freedom of speech!" I would bet the money in my pocket that his intent is that he knows this stuff makes him more money than if he censored it. But he will of course pretend it's about 1A freedoms.
Friction/barrier to entry is the biggest difference. People generally didn't do things like that before due to a combination of it being a colossal waste of time and most not having the requisite skills (or will and patience to acquire said skills). When all it takes is @mentioning a bot, that friction is eliminated.
How is the atomic bomb different than me going to a foreign country and manually stabbing 500,000 people in the throat?
I would say lots of ways. And that's probably why I have a few knives, and zero atomic bombs.
For one liability: the person doing the Photoshop is the one liable for it, and it was never okay to do this without consent.
The obvious problem is that Grok is also distributing the illegal images.
This could be easily fixed by making the generated images sent through private Grok DMs or something, but that would harm the bottom line. Maybe they will do that eventually once they have milked enough subscriptions from the "advertising".
why does it matter if grok is advertising or you are advertising? in reality there's no difference. its just a tool you can invoke.
How is having cameras on every street corner that identify you based on your face and height and weight and gait and the clothes you're wearing and anything you're carrying, or the car you're driving by its license plate and make and model and color and tires/rims and any visible damage, accessories, etcetera, and taking all these data points and loading them into a database that cross-correlates them with your credit bureau data and bank records and purchase history and social media and other online activity and literally every single other scrap of available data everywhere, and builds a map of everything about you and everywhere you ever go and everything you do and have ever done, makes it trivially queryable by any law enforcement officer in the country with or without a valid reason, retains it all in perpetuity, and does all this for every single person in the country without consent or a warrant issued by a judge, different from a police department assigning an officer to tail you if you are suspected of being involved in a crime?
We are going to be in some serious fucking trouble if we can't tackle these issues of scale implied by modern information technology without resorting to disingenuous (or simply naive) appeals to these absurd equivalences as justification for each new insane escalation.
"Technically, anyone can make napalm at home. What's wrong with Walmart selling it?"
> But how is it different from other tools like doing it manually with photoshop?
Last I checked Photoshop doesn't have a "undress this person" button? "A person could do bad thing at a very low rate, so what's wrong with automating it so that bad things can be done millions of times faster?" Like seriously? Is that a real question?
But also I don't get what your argument is, anyway. A person doing it manually still typically runs into CSAM or revenge porn laws or other similar harassment issues. All of which should be leveraged directly at these AI tools, particularly those that lack even an attempt at safeguards.
It circumvents basic laws in plenty of countries giving kids access to these tools.
It could easily be solved by basic age verification.
The csam stuff though needs to be filtered and fixed as this breaks laws and I'm not aware what would make it legal, lucky enough
From Wired's original article, at https://archive.is/https://www.wired.com/story/grok-is-pushi...
> Every few seconds, Grok is continuing to create images of women in bikinis or underwear in response to user prompts on X, according to a WIRED review of the chatbots’ publicly posted live output. On Tuesday, at least 90 images involving women in swimsuits and in various levels of undress were published by Grok in under five minutes, analysis of posts show.
ChatGPT and Gemini also do this: https://x.com/Marky146/status/2009743512942579911?s=20
Real question as I don't use ChatGPT or Gemini: They publish images of women in bikinis or underwear in response to user prompts? Where do they publish them? I'm looking at Gemini and I don't see any sort of social aspect to it. I just tried the prompt "picture of a dog" and I don't see any way that another person could see it unless I decided to publish it myself.
For this particular one it seems to be that you @grok under a posted image with a request for modifications and that account posts the modified image as a reply.
Right, that seems to me like an important distinction. Other people in this thread have said things like "Well you can draw people in a bikini with a pencil without their permission! Should we ban pencils, too!?" Honestly if someone wants to be weird and draw bikini pictures of journalists they don't like AND KEEP IT TO THEMSELVES, then whatever I guess. That's not what this is. Grok is creating the images. Grok is publishing the images. Grok is harrassing the subjects of the images by posting it in their replies. Neither ChatGPT, Gemini, nor pencils are doing that. (And that doesn't even get into the CSAM aspect.)
One of the many reasons I prefer Claude is that it doesn't even generate images.
The moderation is surprising here, I'm not really bother by that, but if it's unclear: I'm just adding context, not endorsing bikinis or disovowing bikinis in the comment above.
Do they post those images to Twitter under their corporate accounts?