A human postmortem of the 1996 AOL outage
ngrok.comIts 1996 - AOHell is loading, Aphex Twin blasts over 28.8, cordless phones hidden, a pizza is on the way…
AOHell didn't take any time to load, and no one was streaming music on 28.8.
I was definitely listening to RealAudio radio stations over a 14.4 connection.
I was gonna say, we absolutely had streaming music. Did it suck? Yes, but it was novel, so it was acceptable. I had a 33.6 connection that my ISP eventually upgraded to full 56K, which I discovered by noticing that the dial-up handshake sounded different.
Man… I sound old.
You’ve unlocked a memory for me! I had that same experience when I heard a different series of squeals than normal only to realise the modem had negotiated something faster than 31.2!
I turned my squeals off. Got a phone bill. Turned on the sqeals. Dialler.exe was calling the Seychelles. That was back in the days when my telecom let me off the bill (7* my monthly income at the time) because 'You have a virus".
Reminds me of a time that on the weekends my parents would drive to my grandads who had a computer. My grandad was a television screen writer and I had discovered Habbo Hotel, so win win.
Of course hormones being all the rage at 17, I decided to look at porn and printed it out so I could show it around at school. What I had done was downloaded a premium rate dialler that ended costing him around £100 back in 2003.
I had gone downstairs and forgotten to disconnect. He's passed now but sorry Grandad for the phone bill. I had never realised what I had done until many years later.
Still, I was one of the cooler kids in school for having a HP DeskJet printed crumpled piece of paper of a naked lady.
in 2003?
Yeah, I was 16. Dial-up was still primarily the internet, 512kb broadband in the UK didn't really hit until 2007/8. The best you had was 128k ISDN.
I used to listen to Art Bell on the Coast to Coast RealAudio stream - from the UK!
I think it was before "RealVideo" so it was still just "RealAudio" and not "RealPlayer". Or something like that.
I'm taking the downvote in stride, but 14.4 and 28.8 RealAudio streaming seems like it lasted all of 3 months and then people had napster and were pulling 96k mp3s.
I had AudioGalaxy before Napster, and preferred higher bitrates (192 KBPS CBR, until VBR took off), but yes - RealAudio didn’t have that long of a lifetime in comparison.
> no one was streaming music on 28.8.
In 2000-ish I was streaming video on 28.8 - not very good video, but video nonetheless.
Also, I'm pretty sure "28.8 Aphex Twin" was more a reference to this sound:
more like over 88.1
Something that I have started doing lately is asking ChatGPT et al to check usenet for reactions from users about events (if it is the right 80's/90's time period). Sure enough, aol.sucks on usenet had some choice words about the outage:
>What does Cisco stand for?? Case's Internet System Crapped Out. That's right, Steve Case and his AOL pig fell victim to some mickey mouse networking equipment. Unfortunatly for AOL, they were the first ISP to feel real pain from using equipment made by Cisco Systems.
https://groups.google.com/g/alt.aol-sucks/c/iqjd7crtPs4 https://groups.google.com/g/alt.aol-sucks/c/K75nltM31Bw https://groups.google.com/g/alt.aol-sucks/c/vVup-HvlPWM
Here's a reporter asking for comments and getting laughed at and trolled: https://groups.google.com/g/alt.aol-sucks/c/mStonlu_H8E
Some more serious reactions over on comp.risks: https://catless.ncl.ac.uk/Risks/18/30#subj2 https://catless.ncl.ac.uk/Risks/18/31#subj3 https://catless.ncl.ac.uk/Risks/18/41#subj3
>Yesterday morning, I got a call because their mail system was backing up heavily. It took a while to discover the cause, but it turned out to be AOL. Because AOL's incoming mail from the Internet runs on relatively slow systems, and because they receive hundreds of thousands of Internet messages a day, they have 30 systems to receive incoming mail, all pointed at from the AOL.COM name. That means that any mail system trying to send mail to AOL would have to individually try all 30 addresses before giving up. Translate that to a 60 second (typical) wait for a connection timeout, and you've got a 30 minute time-in-queue for an AOL message.
nanog on seclists was an interesting read too https://seclists.org/nanog/1996/Aug/51
Flamewar over sendmail not handling outage well > Remember the AOL outage? One host built up a backlog of 2000 messages for AOL---but, because it was running qmail, it didn't even slow down. Meanwhile, sendmail users were choking on much smaller queues. https://groups.google.com/g/comp.mail.sendmail/c/TeNdv2laT94
That's really cool! I actually did download an archive of aol-sucks while researching this, but the software I was using to look through the mbox file was kinda buggy so I gave up. I'm literally the meme of the miner guy giving up right before hitting diamonds.
Yeah I was surprised a while back that ChatGPT was pulling them up (I was doing some research on origins of sudoku and it was pulled up very old threads on Usenet). So now I specifically ask for it and it consistently finds me some gold. Might take a few rounds of saying do deeper research but it often works.
The bit about Steve Schalchlin really affected me. The idea that someone's whole life could have been different, or much shorter, if they hadn't seen a piece of info at the right time. Gives me chills.
In a touch of irony this blog is throwing some null error
It's first paragraph leaves me disappointed. It's 1996. Worries over a tech bubble are a few years away. And there are no tensions with Russia really, because Americans didn't give a damn about the first Chechen war.
Yeah, this article was clearly written by someone too young to have any solid understanding of the time period they're attempting to describe.
For starters, they don't seem to realize that AOL usage at the time was still very much undergoing the transition/hybrid from the earlier "online service provider" model (like Prodigy, CompuServe, etc), with much of its exclusive content only accessible by "AOL keywords" and not being web/internet-based.
The anecdote about the AIDS patient is especially weird, since the linked post does not reference this AOL outage at all -- they explicitly call out their regional ISP, "The Loop", for being down multiple days. And the BBS they reference in another post is almost certainly a dial-up text based system.
So this article's claim that due to this AOL outage, "perhaps that BBS post would have been bumped off the front page by the time he checked" is simply nonsense: this guy wasn't using AOL, and a dial-up BBS wouldn't be affected by an ISP outage, and forums on a dial-up BBS had no notion of a "front page" whatsoever in the first place.
Not to mention, claiming "bell-bottoms are back in style" in 1996, what?
> We, ngrok, have sponsored Mac to write this post because we think it’s an underexplored perspective on the topic of reliability.
Uh, okay. Were there any reliability perspectives gained from this 30-year-old postmortem that would help us in the modern age? After reading the article, I feel the answer is "none". Not that I'm complaining I love this era of the internet. But I fail to see any importance here.
History is important and interesting to some of us. Just because it's not a direct 1:1 mapping of computer problems today doesn't make it any less intellectually stimulating to read about
The human perspective of the outage, and the degree to which that gets buried in a fully-monopolised hellscape.