PhysicsForums and the Dead Internet Theory (2025)
hallofdreams.orgUsing AI to fake posting activity is bad.
Using real member accounts to post the fake AI content is so much worse.
I can’t even imagine the thought process that led to thinking it was a good idea to generate fake posts but assign them to real users. If they were going to fake posts, why not fake the users, too? They turned one bad decision into a much worse decision.
It must be sad to see your proud little corner of the internet decline in popularity, but this move took whatever positive historical vibes it had and lit them on fire.
> the thought process that led to thinking it was a good idea
Seemingly that can be found here: physicsforums [dot] com/threads/official-response-to-allegations-of-ai-misuse-in-pf.1078532/
> During the past several months, to counter declining engagement, I experimented with generative AI, intending to add value to "thin threads", especially those with 0 replies.
> For experiment #3, I used real accounts that I considered to be decade-old spammers as best I could tell, but I can't be 100% certain. I categorically regret and reject such a shameful and unethical decision.
(that website also modifies your clipboard when you copy stuff, big fuck you to the site owner, seemingly lacking any sort of guiding compass, good riddance to them)
But what's the end game envisioned here? Turning the website into an LLM proxy and hoping enough suckers look at your ads instead of those from other sloperators?
Boosting impressions to maintain "passive income" is one possibility.
I’ve seen niche forums in the past insert affiliate links, so when somebody clicks on that HP graphing calculator they go straight to Amazon and the site operator gets a 2% commission. This was the approach of a website that plagiarized Obscure Sorrows and was discussed here on HN (https://news.ycombinator.com/item?id=48611823).
And then there’s tinfoil hat territory: massive influence operations that span multiple forums and Wikipedia and random blogs to support specific points of view about scientific claims, preferred technology approaches tied to commercial platforms, and even historical claims such as who or what developed a scientific concept.
Helping people who have questions about physics
> Using real member accounts to post the fake AI content is so much worse.
Yeah, that part seems not just ethically bankrupt, but also really really stupid and counterproductive?
It all made a bit more sense to me when I learned that the guy who did all this is not, in fact, a physicist like I had originally assumed, but rather an "SEO lead at Shopify". [1]
Brings to mind something I read recently and can't stop thinking about: "The Internet is a store, but it used to be a library."
everyone involved in seo is a ghoul poorly masquerading as human in my book
Like other zombified sites, it exists to collect ad revenue from Google. It's amazing how despite 25+ years of technological change, that Google's decades-old "click ad" model is stronger than ever.
Previous discussion:
https://news.ycombinator.com/item?id=42816284 - Jan 24, 2025 | 154 comments
This comment above is date mangled due to HN second chance pool <https://news.ycombinator.com/item?id=26998308>.
It's ironic because the article (and many comments in the previous submission) are about how one shouldn't mess with user data to show fake engagement!
As the percentage of LLM-generated stuff in the world increases relative to natural stuff, I don't see how it's possible to continue to train new models indefinitely. At some point, won't LLMs be trained on a corpus that is majority LLM-generated data, with everything that entails both good and bad? Seems to me that singularity is just going to kick off a self-reinforcing spiral... Bigger minds than me have to have considered this, is this a legitimate concern?
I wonder if someone has redone the experiment of retraining LLMs on LLM output that was conducted a few years back that showed a pretty drastic degradation in quality. I imagine it's hard to perform on the closed source frontier models and the ballooning cost of training doesn't help.
In theory as the models improve their ability to ape human writing the effect should diminish, though iterative reingestion of their own output is probably still detrimental long term.
The training corpus of pre-LLM writing is like iron from before the development of nuclear weapons
Maybe with time we'll accomplish the same thing that's happened with pre 45 steel and the instruments will need less of it. I guess in this analogy it'd be better quality filters that can detect and remove LLM-spam.
How do humans deal with this?
For millennia we created documents supporting the geocentric model, all wrong! How'd we break out of that rut?
I don't know if AI are currently equipped to deal with this as humans did, but maybe they can be?
the models can also get better at using known data, not that it always needs more data . This was the case with math and coding.
Claude reports that this is called "model collapse" and assures me Anthropic is on top of it
The current improvments in LLM performance are not driven by feeding even more text from the internet into them, in fact they're on the whole deliberately being trained on LLM generated text, with some reinforcement learning through a feedback mechanism (generally success on some task which requires some intelligence but is easier to verify than do). So it seems like it's not really going to be a bottleneck, though I think this RL process is part of why the frontier LLMs write so weirdly.
Jesus. I'm risking identifying myself here, but it has happened before. I was one of the earlier moderators of PhysicsForums back in 2003 and one of the younger ones. I think myself and one another was an actual student, whereas the others were all professors or retired. I believe my last login was 2013 and last real activity well before that. I no longer have access to the email used and couldn't recover the account even if I wanted to.
I only talked to Greg remotely and never met him. This is a huge betrayal of trust. Older moderators have died since then. This is a very longstanding community. Even if we didn't stick around forever, many of us became real-world friends. Adding autogenerated summaries is one thing, but inserting comments in a real person's name that they never made, when you have no knowledge of what became of that person, who may even now be dead, is beyond the pale. You're rewriting history. The fact that this site still exists at all is extremely gratifying. It's a huge part of what has made me who I am today. It's a big part of why I don't really pontificate all that much, don't have a blog. I read what I wrote there nearly a quarter century ago, and it's not all bunk. A lot of what I said holds up. Plenty of it was raw factual data never in dispute. But when it came to politics and economics and forecasting of the future, I said things so wrong and thing I so strongly disagree with now that having that record teaches me how flimsy and full of shit so many of our opinions are, no matter how smart we are and no matter how strongly we hold those opinions. I've 180'd things I was completely certain of during the first Bush administration. Heck, you can add voting for Bush to that list. There is no way in hell I would do that today.
History matters, Greg. It can both humble and enlighten us. But only if it's true.
The one sane comment in this thread.
> Adding autogenerated summaries is one thing, but inserting comments in a real person's name that they never made, when you have no knowledge of what became of that person, who may even now be dead, is beyond the pale. You're rewriting history.
> History matters, ... It can both humble and enlighten us. But only if it's true.
Reminds me of George Orwell's writings. Specifically Notes on Nationalism - https://en.wikipedia.org/wiki/Notes_on_Nationalism Here "Nationalism" is to be interpreted as more akin to "Tribalism" i.e. defending one's group at all costs and against all evidence/facts/truth as needed.
All nationalists have the power of not seeing resemblances between similar sets of facts. ... Actions are held to be good or bad, not on their own merits, but according to who does them, and there is almost no kind of outrage – torture, the use of hostages, forced labour, mass deportations, imprisonment without trial, forgery, assassination, the bombing of civilians – which does not change its moral colour when it is committed by ‘our’ side.
So, how did all these hacked accounts or backdated posts happen? The article never says that even though the author talks to the site founder and admin.
PhysicsForum's admin admitted creating them shortly after: <https://www.physicsforums.com/threads/official-response-to-a...>
It's heavily implied the owner had them generated by a bot, potentially as part of his day job in SEO
Completely forgot about PhysicsForums.
Also this is a good example of the trend that has already been ongoing on HN since 2022. I think one other long-lasting HNer is working on a similar analysis of HN at the moment.
Edit: can't reply
> Appreciate some more details...
It's their story to tell. Not mine.
I haven’t noticed a huge change since 2022, except perhaps that politics are discussed quite a bit less
What is this "trend" and "similar analysis" you mention? Not clear from your comment.
Appreciate some more details.
I have an ongoing analysis and here's some highlights so far: https://news.ycombinator.com/item?id=50225117 tl;dr things don't looks good
I am not quite sure what the HN owners/mods are doing about it, if anything at all.
Due to non-linear effects of propaganda, a single/small-group of people working in a concerted manner can easily change group dynamics specifically its "fairness perception" and thus ultimately lead to the detriment of the group itself.
I don't know how much they can do. I don't want to speak for the mods, but most of the problems on this site come from upvotes and downvotes. I guess some cheap things you can do are increase the karma threshold required to downvote and apply a smoothing function to upvotes (like quadratic voting for vote scores, aka n votes on a comment bumps the score up by sqrt(n).)
But ultimately a site moderated by up and downvotes that wants to keep a relatively light hand on moderation is going to end up like what HN is today. Changing the voting structure on the site or the moderation POV on the site would be a radical change and could change the entire site's culture drastically.
Oh, HN owners/mods can do a lot if they wanted to. There is a lot of propaganda/influence peddling/public opinion shaping/shilling/obfuscating etc. going on here which has only increased after the AI boom.
The stock answer to the above charge is that "this is more of the individual's perception and not reality" and "HN is just representative of what is going on in the society". Also;
Ask HN: What is Hacker News's business model? - https://news.ycombinator.com/item?id=23671531
Humans will game everything they can to gain an advantage even if it were only for silly imaginary things like "upvotes". Add in our psychological propensity for "Tribalism" and you can see we have to be a bit paranoid when it comes to social networks. Introduce anonymity into the game and things get even more worse.
So it behooves the HN owners/mods to have some sort of system to analyze everything and do some inferences for insight even if they don't act on it immediately.
Some users have already developed some models and shared it on HN; eg:
1) Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun - https://news.ycombinator.com/item?id=40307519
2) Mapping Hacker News to find who knows what in the HN community - https://news.ycombinator.com/item?id=41069527
3) Show HN: Hacker Atlas - A map of what Hacker News talks about - https://news.ycombinator.com/item?id=49844497
4) Hacker News Data Map - https://news.ycombinator.com/item?id=42035981
PS: There are a bunch of books on "Social Network Analysis" which one can consult. A good one is the book of the same name by Tanmoy Chakraborty.
Say more?
I've been reading HN since 2013 and the quality has gone down, kind of like eternal September.
2022 was when the degradation really took hold.
HN essentially became mainstream after the New Yorker profile, Reddit's degradation due to the mod blackout and API changes, and Lemmy flopping.
I've gotten my HN back to the pre-2022 consensus but it takes severe filtering, with around 200 posts being hidden a day. And it's those posts that drive the majority of discourse on the platform now.
The fact that throwaways like you are so normalized is a huge reason.
HN has also gotten divorced from tech and water-cooler conversations in much of the tech industry, and a lot of that seems to be because HN no longer has the same plurality of Bay/NYC/Seattle/Austin techies that it used to, and the userbase is way older than much of tech now (I was always younger than average on HN, but the amount of 1980s to early 2000s references shows that the platform has frozen in time demographically).
Ignore Reddit - HN has functionally become Slashdot (the fact that I even know what slashdot is highlights how the age demographics are so skewed on HN).
Edit: can't reply
> 2022 is when tech funding dried up...
Tech funding didn't dry up then. It just coencided with the time period that most vintages that originated in early 2010s were closing out and those that originated in 2016-18 were hitting their 5 year mark.
This meant a lot of dry-powder didn't get deployed until 2023-24.
> stands to reason that we ran out of hacker topics...
They are still being posted at the same rate based on my filtering. The difference is a massive deluge in non-"traditional HN" related content being posted, and the younger generation of techies and founders (those below 35) not using HN.
2022 is when tech funding dried up, so it stands to reason that we ran out of hacker topics. The odd person still doing something cool in their basement for fun does not supply a sufficient stream of curiosities to keep the feed full. That required the unique time (ZIRP, etc.) when people were paid handsomely to explore cool things day in, day out.
HN was a product of its time.
--
> They are still being posted at the same rate based on my filtering.
"I rewrote X in Rust" isn't a hacker topic. Being able to draw some kind of connection to the tech isn't the same thing.
That's a good point. Fwiw, I use throwaways because I don't want certain types of users to know anything about me. I wouldn't have felt this way, say, 5 years back.
What tools do you use for filtering if I may ask?
I built my own system using both sentiment polarity analysis, hard-coded logic, and some semi-active tuning, but I'd rather not divulge specifics. IMO anyone can spin up their own similar system.
> I don't want certain types of users to know anything about me. I wouldn't have felt this way, say, 5 years back...
Frankly, most people DGAF about random HNers and (a general or meta-comment targeted at the reader, not you OP) if someone needs to use throwaways to post negatively polarized statements on HN, then they shouldn't even be on HN in the first place.
> most people DGAF about random HNers
It is never most people you need to worry about.
> if someone needs to use throwaways to post negatively polarized statements on HN
If it were only read by other HN users who understand that a user account is not a human then it wouldn't matter, but the world is not so simple. There are people, who do not understand that forum accounts are not humans, who are working hard to associate internet activity with humans and will use it against them.
Of course, we already know how to identify individuals by their writing style, so a throwaway account probably isn't the screen the OP hopes it is. Still, it is the best source of making it as difficult as possible we seemingly have.
The "Dead Internet Theory" is too sad and too easy. I'd propose the "Dead Society Theory", the history of the Internet from the late 2000s to the late 2010s left people socially scare, paranoid, angry and hopeless about the possibility of communicating online.
Which is to say the Internet can be "dead" without being filled with undetectable or detectable fake posts, not that fake posts help things.
IMHO, the theory is wrong. There is tons of real, human interactions online. I see it on reddit constantly. The bot activity is easily ignored or algorithmically suppressed most of the time.
If the dead internet theory were true or real, Google and Facebook would not generate so much revenue. Advertisers would notice that they are not getting any conversions and stop spending.
> There is tons of real, human interactions online. I see it on reddit constantly.
Are people responding to submissions, reposted by bots, a genuine interaction? It is one kind to know there are some bot accounts on reddit, and other to experience it first handed. For example, recently some karma farmer posted half of original post, and comments that appeared under it still referenced second page, because entire threads also got copied verbatim from previous submission, using different accounts.
It really feels like reddit admins don't do enough to combat that behaviour. Maybe, just like admin of PF, they think fake activity is better than none at all?
I'll add to the dead bit:
Many years ago I did a search for my name and turned up a LOT of things I did not actually write. This wasn't LLMs, this was scrapers doing a poor job of scarfing up old content--comparing what I saw vs my recall of the discussions there was a lot of stuff that lost the quoting, words were attributed to me that were actually quotes I was responding to.
It’s especially disappointing to see this done in such an unnecessarily dishonest way. If the project truly wanted to add value, I can imagine interesting AI uses that don’t lie or trick people. You could post a correctly-dated AI Answer (and prompt it to use the web search tool and cite sources) to posts that apparently never got one, and credit the exact model used, which would help future readers decide how confident they might be (a GPT-3 answer vs a GPT-6 Astra answer, for instance, are very different).
Heck, you could even engagement-farm if you have to, by taking science stories in the news and say, generating a poll to try to spur discussion.
But what they actually chose to do is super gross, helps nobody, hurts the site’s reputation, and even devalues the site tremendously as possible training data for AI. (Which, some AI detractors might actually be happy about, but I think isn’t really a good thing.)