Settings

Theme

Highly classified US national security communications network goes down

cnn.com

602 points by monkeydust · 735 comments · 1 min read

Reader

https://www.reuters.com/technology/highly-classified-us-nati...

96 threads
drakonka

I am trying to get some grasp of whether all these seemingly major incidents are actually happening at higher rate or if it's always been like this and the difference lies in how/if they are reported or whether I'm paying attention.

usernomdeguerre

People seem to be leaning on the AI speculation from the "vibe coded" angle, but couldn't this also be an "AI is the reason" from the attacker side? I think we've seen more than a fair share of AI assisted exploits that chain many small vulnerabilities into something more pronounced. I'm thinking of the rash of attacks in SK and China against Banks using offensive harnesses.

I'd be more willing to believe the network in question here was under an attack load that either was probed or exploited far enough that shutting it down was worthwhile. It seems more reasonable to me than believing they've let AI run amok on their internals such that it's now failing.

Not because I'm deifying the NSA but because the system has to be decades old, at least, and I wouldn't expect significant changes to something that "just works".

  • testtestmetest

    If they were attacked, they wouldnt be calling the WSJ and screaming about it. We're being fkd with, theres some sort of strategy of tension regarding cyber/fraility of systems being pushed.

    • sedivy94

      Couldn’t agree more. The cacophony of significant events happening across the globe at any given moment is impossible to distill. National news is not a good-faith sampling of that pool, but rather a mindful selection. Pre-programming is real and frequent.

      • 0xDEAFBEAD

        So many unfalsifiable vague-ass conspiracies on this board.

        Everyone sees the incompetence of the Trump admin. Everyone sees things like this highly classified network breaking and the Signal leaks -- it's not out of character or anything. But when it comes to "pre-programming" the population, the powers that be suddenly turn hyper-competent.

        • cucumber3732842

          Nobody is alleging conspiracy, just that enough incentives align to produce comparable outcomes.

        • testtestmetest

          The people that run the NSA are non-partisan careerists.

          Keeping telling yourself Trump has any say in how their organization functions at an operational level. I think Trump is incompetent but I dont think that incompetency travels down to the operational level in intelligence like you think, the politicans trust these guys to do their jobs with their expertise.

          You can call people conspiracy theorists all you want as an insult (thats what it is), or you could just try have an analytical understanding of the past and how these institutions operate.

          • 0xDEAFBEAD

            >Keeping telling yourself Trump has any say in how their organization functions at an operational level.

            "CIA Employees’ Names Risk Exposure After Trump Demands List in Unclassified Email"

            https://www.newsweek.com/trump-unclassified-email-cia-agents...

            What would falsify your theories?

            The thing about them being "nonpartisan careerists", at least, appears to be false. See https://steadystate1.substack.com/

            Same for "the politicans trust these guys to do their jobs with their expertise". See https://www.youtube.com/watch?v=7UD6F4F-Bcw

            So roughly speaking your claims seem to fall into two categories: (a) unfalsifiable, or (b) falsifiable, and additionally false. If your falsifiable claims are false, I expect the same is likely true of your unfalsifiable claims.

            >You can call people conspiracy theorists all you want as an insult

            How do you explain, for example, people who insist that the earth is flat despite all evidence to the contrary? There's clearly a cognitive trap around conspiracies which a lot of people fall into, and I'm supposed to avoid mentioning it when it comes up, in order to be polite? No thanks.

            What does Occam's Razor say? https://substack.com/@kryptogal/note/c-390204515

            • testtestmetest

              Did you really link a video from. The fake grifter CIA guy?? That guy is like the course seller of spy youtube. Completely lacking any credibility and has been called out by dozens of people for probably lying about all his experience.

              Im talking about the NSA anyways, I can assure you the NSA is very scientific and an apolitical institution. Much closer to NASA than it is the CIA.

              Why are you comparing me to a flat earther.. You're too irrational and judgemental to even have a discussion about this. You're one of those people who has to be the anti-skeptic, its kinda cringe rationalist speak where you think the world is black and white.

              Im not going to argue with someone who resorts to these tactics to discredit someones argument just because they dont like their takes.

    • m000

      > If they were attacked, they wouldnt be calling the WSJ and screaming about it.

      They might as well do just that. I've heard the theory of a "soft false-flag". I.e. something that will not have casualties and will not inspire mass panic, things which could backfire. But it should be visible enough to DC circles, so it can be used by people that already have access to the administration to nudge Trump into making certain decisions.

      Sounds a bit like 5-D chess, so I'm not buying into it for now. But an interesting theory, nonetheless.

  • alfalfasprout

    Both can be true. Systems that are becoming less rigorously built by less qualified folks (or teams to[ small to manage the pace of change) being pushed to use AI for everything are also easier to attack. Combined with AI being used to identify and execute new attack vectors that's a recipe for disaster.

getcrunk

as other commenters have said:

> theres some sort of strategy of tension regarding cyber/fraility of systems being pushed.

> 'Manufacturing Consent', new Congress AI regulation for industry capture in the pipeline

both capture my inclinations, but the other thing is this can simply be symptoms/indicators. looking at the recent faa thing, this could be "move fast and break things" happening in real time.

new contractors, new programs, more outsourcing/privatizations etc, palantir type companies moving into new places.

this is right after midterms, so the admin is doing its final phase of plays

  • DrScientist

    There are also other things that affect what's reported in the main news versus places like the theregister.

    ie it could just be that this sort of thing happens all the time, but is just getting reported more on places like CNN et al due to heightened sensitivity to the issue.

    I suspect it's a combination of all of the above:

    - never been easier to hack

    - people with agenda's pushing stories

    - people with careers reporting the zeitgeist.

    The one narrative I tend not to believe is the foreign adversary one - as if I was a foreign power and I knew, for example, how to bring down the US national security network - I'd be saving that till I really needed it.

  • clivedup

    I expect organisations to be pretty effective at stonewalling disruptive change, especially if they perceive the stimulus will go away soon.

  • andai

    Honestly all this crap falling over now instead of in a much worse context is probably the best possible outcome.

    • tty456

      That is assuming the admin acts appropriately at this moment and hardens security in all its agencies

twotwoone

Don't fall for stories that AI did all this. The NSA's internal network is unavailable every year or so. Just like all other services.

The only thing that distinguishes this particular moment in time is the thousands of bots from companies preparing for an IPO.

tgsovlerkhgsel

That's a LOT of things going down, and I haven't heard of some major infrastructure outage likely to affect many sites in this manner.

Coincidence or is there some common explanation that I missed?

  • DamnInteresting

    A single data point: I run a fairly well known site, and in the past two months my median daily traffic has quadrupled. Essentially all of the new hits are probes for vulnerabilities, presumably by AI agents.

    The site has pretty robust defenses, so nothing has broken through yet, but a penetration or an effective DDoS feels inevitable as the probing steadily escalates. I expect that juicier targets are seeing a lot more of this kind of activity, possibly leading to some of these outages.

  • bhouston

    There is a lot of things going down.

    I think the opportunity right now where we have powerful AI hacking systems but not enough protections is to establish long-term presence in all networks of interest. Not bring them down.

    • colinrand

      Completely agree. Things are moving too fast and so are breaking.

      For example, we updated a GCP managed base image of Ubuntu that was released with 350 CVEs patched, some were as high as 9.8. In doing so, they (GCP) changed the DEFAULT setting for hardware GRO from off by default to on. That's a massive networking change packed into a security update.

      We only caught it AFTER it started causing issues in production. We didn't have the OS pinned because (decently valid reasons) or catch it in non-prod (decent reason + human mistake).

      This is on a modern highly automated tested resilient well architected platform.

      I can only imagine these other platforms are dramatically more exposed that we are, hence these common large scale outages.

    • pianopatrick

      That assumes that people did not already have long-term presence in networks of interest.

      More plausible to me is that spy agencies had previously found and used some of these hundreds or thousands of vulnerabilities to hack into those networks just fine. But now other people like cyber criminals are using these vulnerabilities via AI. That is messing up both the attackers who had already gotten in quietly and do not want the attention and the defenders who now have to respond to a noisy intrusion they cannot ignore.

    • daheza

      None of these have been due to hacking? It has all been just standard issues probably increased by cost cutting measures.

    • cj9

      Also, anyone anywhere that horded/designed exploits will be wanting to use their backlog of these "assets" before systems are patched. From criminal networks to nation states.

      Outages could be grey-hat testers and emergency updates.

      A temporary period like we are experiencing is to be expected when moving to a time where a single engineer can perform the equivalent work of a team of excellent engineers working for weeks. Security has always been a game of who has the capabilities, means, and requirement for any given attack surface.

    • Florin_Andrei

      We also have powerful incentives to deliver more "value to the shareholders", while using AI code generators we don't fully understand.

  • postsantum

    Appstore connect is also down

  • wat10000

    It reminds me of the aftermath of the DCA crash in 2025. For a few weeks it felt like planes were falling out of the sky as a bunch of (sadly somewhat commonplace) non-airliner crashes got national attention they otherwise wouldn't have.

  • chabes

    What do people expect when top government offices are occupied by former Fox News hosts?

  • winddude

    Elon Musk and Trump with DOGE. They cut ~300k federal workers, cut governments contracts across all departments, including the FAA, NSA, home land security... those organizations have been raising alarms since that the cuts to civilian contractors etc would lead to outages and issues since, including pentagon, DoD, etc. But you know Idiocracy.

    • edgyquant

      Bloated government agencies want to justify infinitely more tax dollars, that doesn’t mean we should treat them as unbiased actors nor the people who want to blame making changes to the government for every problem. People have to be able to elect representatives who make changes to government agencies without being treated like doing so is evil and stupid.

dustfinger

I may not understand expectations, or the law around this, but I find it a little surprising that the NSA would publicly announce that their internal and highly classified network is down. At least not before getting it back up, running and secure again.

  • King-Aaron

    The government is intentionally trying to highlight things as being broken, as justification to sell and privatise infrastructure.

    • esalman

      Everything will be privatized and people owning AI companies like OpenAI, Anthropic, SpaceX etc. will get the contracts. Microsoft, Amazon, Google will be happy selling infra, and Nvidia will be happy selling gpus. This is the worst outcome of capitalism.

    • Gagarin1917

      The Dems are going to pass privatization bills?

    • nojvek

      Anthropic gonna get their regulatory capture card. Ban open source AI.

      • King-Aaron

        Yep. "Oh no open weight models hacked our super secret FBI server (small router that runs the break room spotify speaker) so now we must ban all chinese models"

        • 0xDEAFBEAD

          Things the Trump administration can't do:

          * Keep conversations secret on Signal

          * Make over the Reflecting Pool

          * Avoid firing critical employees during DOGE

          * Retain IT staff

          * Calculate tariffs using a sane formula

          * Label African countries correctly

          * Etc., etc.

          Yet we HN readers are also supposed to believe that there is a galaxy-brained plan involving the US government coordinating with Anthropic (Anthropic, the same company the DoW designated as a supply chain risk) in order to ban open-weight models (any domestic ban would only affect US corporations and do essentially nothing about the global cybersecurity environment).

          What is motivating these mental gymnastics folks?

          • mobelkh

            there is no need for a big brain plan, something happens, corpo lobbies use it to push talking points to get closer to regulatory capture.

            if they did have a brain they'd know this is terribly short sighted

          • King-Aaron

            > What is motivating these mental gymnastics folks?

            Maybe because all the western AI leadership is saying and pushing for it fella. I tend to feel the mental gymnastics demonstration is coming from you currently.

            • 0xDEAFBEAD

              I'm willing to believe that the people speaking against open-weight are speaking against open-weight. I'm skeptical of these fantastic conspiracies.

              Consider Occam's Razor. What is the simplest explanation? Some ideas:

              * These are issues due to people updating software which hasn't been updated in ages, as a measure to guard against cyberattacks.

              * These are cyberattacks, and the people attacked don't want to admit it: https://substack.com/@kryptogal/note/c-390204515

              Imagine if you lived in New Orleans and a civil engineering firm is saying the levees need to be upgraded in order to deal with hurricanes. You see a hurricane on the horizon and think: "I bet this weather was orchestrated by our local civil engineering firm in order to drum up some business."

              • King-Aaron

                I feel that you are intentionally trying to discredit any argument because of your preconceived notions about the benefits of AI or whatever, so all you're able to see is people calling out conspiracy theories. Apologies if that isn't the case, but it's how it's coming across.

                What I am saying is that these problems - be it intentional or not - are opportunistically being used by the administration to push for privatisation.

                I would wager that the last two years of actions by the party have resulted in an unstable infrastructure situation which they knew would result from pulling funding/oversight bodies/etc. This is also supported fully by their own Project 2025 discussions.

                This is also the very simplest possibility.

  • netsharc

    In a spy movie world, someone deep in enemy territory would be relieved to see the Reuters headline, after failing to reach the NSA network and having no other channel to ask (picking up the phone to call the US embassy would get your door busted by the local men in black).

    I imagine, for the paranoid spy, not being able to call home could also mean "the enemy has severed our comms, we've been compromised, burn everything and run!".

    • anigbrowl

      That sort of communication is necessary for sure, but I'd imagine int he world of espionage some sort of coded false news report would be more likely, eg 'batboy found after 5 hour panic search.'

      Of course, it might be a limited hangout to soften the blow of actually being compromised.

      • bux93

        Apparently, the UK navy check if Radio 4 is still broadcasting. If all communications are silent, including Radio 4, they open their "Orders Of Last Resort", which may instruct them to launch nuclear missiles.

        https://www.forcesnews.com/services/navy/nuclear-promise-let...

        "Submarines on patrol were reported to have briefly gone on nuclear alert in 2004 when Radio 4 went off the air for 15 minutes due to a power cut."

      • netsharc
        • karmakurtisaani

          Can you elaborate a bit how a novella length blog post relates to any of this?

          • SamBam

            I expect GP thought that Borges stories -- or at least a couple ones that are very famous to many old nerds, like The Library of Babel and The Garden of Forking Paths -- are common enough knowledge in a place like HN that the reference to it was enough.

            It would be a bit like linking to Das Kapital in a discussion about class warfare, without expecting anyone to read the whole book at the link.

          • qphe95

            He is saying that in the spy world due to over classification people end up living in their own little different realities and what the commenter above says happens in some of them. Hence the many-worlds.

          • ultrattronic

            I don’t think Borges identified as a blogger

    • sans_souse

      Actually I was just reading about the Thule Air Base Incident, and how that and several other incidents leading up lead to the following quote from

      >

        "Scott Sagan, a political science academic and anti-nuclear writer, postulated that if the HOBO 28 monitoring aircraft had crashed into the BMEWS early warning array instead of Baffin Bay, it would have presented NORAD with a scenario (radio link to "Hard Head" aircraft and BMEWS both dead, no nuclear detonation detected) that also matched that of a surprise conventional missile attack on Thule, leaving the unreliable submarine telecommunications cable between Thule and the US mainland as the only source of information to the contrary."
      
      https://en.wikipedia.org/wiki/1968_Thule_Air_Base_B-52_crash
    • daveevad

      But what if that spy is actually a double-agent!

      So much reporting about clandestine activities falls apart because the traditional 5 Ws of journalism cannot be attributed.

    • qphe95

      Thanks for the explanation! Good thing spy movies are just movies and not real!

  • kube-system

    The initial news reports were not from officials but "people familiar with the matter". Practically speaking, once the story is already circulating, officials have the decision between:

    1. say nothing in an official capacity and let people speculate

    2. make an official announcement to head-off any concerns and speculation

    • mrandish

      > The initial news reports were not from officials but "people familiar with the matter"

      Anytime that's the case, always ask "Who wants this headline to be in the media and why?" Note that a credible media outlet won't run a story unless they can confirm the "people" leaking are in an official capacity to actually know the leaked info.

      For example, per later reporting in NYT & WSJ, the "Jensen's sleeping with alleged Chinese asset" leak was from the China AI Hawk faction in the White House (Defense, State and Treasure Depts) trying to undermine the AI Trade Pragmatist faction (Commerce Dept, AI advisors and NVidia) just before this week's summit. That matters because the summit will determine whether Trump extends the H200 card waiver past January. Jensen was effective in swaying Trump at the last minute to get the waiver last year. The well-timed leak of the 6-month old Alice report ensured he wouldn't be on Air Force One going to this summit like he was last year. The reporters who wrote the original story certainly knew this was why it was being leaked but the 'rules of the game' are that the reporter doesn't include the obvious purpose behind a leak in the story (or they won't get the next leak).

      Today's leak may be as mundane as NSA / Homeland Security budget squabbles. When you can't get your infrastructure budget approved, highlighting an infrastructure failure can change that.

    • joe_the_user

      I can't imagine the public is hanging on word concerning whether a network we have never heard of is operational. "Anonymous sources state unknown network nonfunctional" - the world is agape.

      • thephyber

        This take is boring and uninformed.

        You could use this logic about any topic that doesn't directly affect every single citizen in a way they recognize.

        Not every person needs to know or cares to know about every news event. But that doesn't mean the news shouldn't be reported.

        Also, those of us who lived through Sept 11 and the failure analysis + Patriot Act know that the federal government has a "secret network". The reason Sept 11 was successful is because our "secret network" knew about suspicious activities of the 19 hijackers but didn't have any way to message the domestic law enforcement (the non-secret comms networks of the US) to surveil them.

        • autoexec

          > The reason Sept 11 was successful is because our "secret network" knew about suspicious activities of the 19 hijackers but didn't have any way to message the domestic law enforcement (the non-secret comms networks of the US) to surveil them.

          Phone lines were still operating. What exactly stopped them from just calling local law enforcement and saying "These guys look like they might be trouble, I can't get into a lot of details, but can you maybe keep an eye on them?"

        • fn-mote

          > but didn't have any way to message the domestic law enforcement … to surveil them

          Isn’t this more usually stated in bald terms like “no communication across silos”, just like in corporate America?

          • thephyber

            That's how Congress people tried to communicate how it worked before Sept 11.

            But the classified system is still a silo, so all of the changes didn't fix anything according to that analogy.

            There were more complicated changes (FISA Court, unclassified tags on classified records, Fusion Centers, Suspicious Activity Reports, etc).

      • kube-system

        It’s not about the specifics. The public and especially other parts of the government would be interested in confirmation that a part of the government can or cannot operate as normal.

        Likewise, nobody gives a shit about Juniper routers in Arkansas but they sure as hell care if the FAA can’t operate.

  • thinkingemote

    I wonder if number stations increased in whatever they do during this downtime.

    • derefr

      I presume at least not in any observable way; Traffic-Flow Security (i.e. filling networks up to a constant bitrate with dummy traffic to prevent traffic fingerprinting) originated in military data-protocol design.

  • testtestmetest

    Its because we're being played with this week. Strategy of tension in terms of cyber/state of things. Buckle up, someone has plans

    • causal

      Okay umm where is the seatbelt you're referring to

      • ghurtado

        I do believe it is somewhere near the buttercups, if I remember the saying correctly.

    • chairmansteve

      I would expect more media hysteria if this were a strategy of tension....

      Am I missing something?

      It feels more like downplaying of real issues (I won't say attacks...).

      • testtestmetest

        My mother who doesnt know what ChatGPT is, is calling me up asking if AI is hacking the government. There's definately media hysteria. Strategies of tension are much more subtle, and typically imposed over a period of time via many events.

      • King-Aaron

        In the project 2025 document, they proposed causing some kind of drama with the FAA as justification to carve up and privatise it.

        Now, there's been some kind of drama with the FAA, and they have already proposed laws to break it up and privatise it.

        This is all project 2025 stuff. They want to disrupt trust in all public institutions as a justification to write up big contracts for their wealthy benefactors to save the day.

        • eks391

          > they proposed causing some kind of drama

          This felt sufficiently conspiratorial that I did some digging. In project 2025 pg(s) 632-633, it does discuss FAA and that certain missions of the org cannot modernize or innovate thanks to the nature of being so heavily regulated, and those portions are better off being removed from the FAA and letting the private sector pick those up, or allowing the FAA run like a business.

          It does not however propose any sort of drama or imply any shenanigans to reach the result. It gives 3 potential COAs to solve the problem, all of which are regulatory in nature.

          • autoexec

            > it does discuss FAA and that certain missions of the org cannot modernize or innovate thanks to the nature of being so heavily regulated, and those portions are better off being removed from the FAA and letting the private sector pick those up, or allowing the FAA run like a business.

            Yeah, there's no plan in the document for creating drama to justify privatization, but they're not shy about their feelings on the subject.

            Any exercise in government-wide budgeting and reform should review the department with an eye toward consolidation, elimination, or privatization that examines the efficiency, effectiveness, and underlying philosophy of each individual component.

            the only real solution is for the national government to do less: to decentralize and privatize as much as possible

            • tosapple

              that would've classified it a terrorist document/conspiracy.

              edit: in reference to drama/shenanigans wrt faa.

          • King-Aaron

            If you have been paying attention to the whole project 2025 story, a lot of these things are discussed by the authors/project owners/etc that complete the picture.

            Yes, this is a conspiracy, in the literal definition of the word. These people are conspiring to take control away from the government and place it in the hands of private enterprise.

        • AtHeartEngineer

          it's funny because it sounds like L3Harris is to blame, like maybe major FAA infrastructure shouldn't be contracted out

    • rsoto2

      War drives profits. Why isn't the insanely unpopular Iran war ending? Money. All of these "outages" can be blamed on Russia, Iran, etc when the time is right.

      Anybody that doesn't understand there is a classified plan to internally destabilize whether through legitimate exploits or planned accidents is woefully unaware of the history of the DoD and CIA.

      https://en.wikipedia.org/wiki/Operation_Northwoods

      • tosapple

        we can't end it, our intent was to deny science. iran's post initiation choices reinforce the logic and assumptions that started it, even if there were no wmd's. if the leak about their intent or deliberation about using a dirty bomb in washington is true then they sealed their fate. the only way out is through, they might be able to 'turn over' those who led the discussion or encouraged it to us for prosecution as terrorists, that may help their case buy we still absolutely must put our hands on the hexaflouride.

        the physical blockade is a brilliant use of presidential power and appropriate, in my book it doesn't constitute war.

      • testtestmetest

        You're being downvoted but I think people need to take some time and become a little more familiar with the history of who's in charge of these institutions, and the things they've been willing to do in the past.

        Its either that systems are starting to decay because everyone is rapidly offshoring all the hard parts of their jobs to mediocre chat bots..

        Or they're actually trying to cause some sort of public stress about the frailty of systems.. They dont go around publishing that their critical systems are going down. It is odd.

        • rsoto2

          Based on what I've seen today everyone on HN is still in their grade school history class where America can do no wrong. Meanwhile DHS is openly posting subvertive neo-nazi propaganda and caging immigrants and recently killing green card holders.

          • mschuster91

            multiple things can be true at the same time. and we indeed are in a perfect-storm scenario of unpredictability.

            the us president is completely demented by now and half of his admin are fascists rotten to the core. domestic and foreign interests both want him to continue war with iran, while other domestic and foreign interests want him to stop that war at once. in less than two months, democrats will take back congress and likely go for cleaning up the us admin at day 1. geopolitical enemies of all kinds want to destabilize not just the us but everyone for their own gains (i.e. russia wants ukraine and china wants taiwan - both impossible as long as the major western countries roughly are on the same page). and ordinary cybercriminals plus for-the-lulz-skiddies now have access to cyberwarfare tooling with open-weight ai models and harnesses.

    • washadjeffmad

      Times this week I've been asked if the Microsoft outage, FAA grounding, and banking downtime were related: Too many

      Times someone has brought up the midterms: None

  • Akio_

    Law wise, every bit of classified information has to be justified as damaging to national security if publicly released in order to be classified. Most non-critical classified networks can't justify classification for the status.

    I find it more likely that NSANet status is not classified than I do that 5 people committed a felony to tell CNN this information...

    • sterlind

      it's very common for administrations to do 'authorized' leaks, whenever they want to maintain plausible deniability.

      • thephyber

        I think not enough people recognize that lots of "leaks" from the Feds are commonly just people who actually are authorized to speak on the subject claiming that it is "background information" so that their name doesn't get reported next to it.

        There's a reason Jeffrey Goldberg was in someone's Signal contacts from the Situation Room, even if he wasn't supposed to be on the big group chat.

  • s_dev

    It's probably not down, probably some honeypot announcement with some other agenda. Like announcing to the neighbourhood you're going off on holiday then staying next door to see who comes up to the property to investigate while 'you're gone'.

  • luxuryballs

    'Manufacturing Consent', new Congress AI regulation for industry capture in the pipeline

  • ExoticPearTree

    If it is a service used by many people, the fact that is down will leak pretty fast. Better get ahead and tell everyone.

    > At least not before getting it back up, running and secure again.

    You’re confused a bit: there is no correlation between being down and being secure. If a device is not operational (for various reasons) it does not mean it is not secure.

  • kQq9oHeAz6wLLS

    Honeypot. Make the bad guys think you're weak and see who comes out of the woodwork to take advantage.

    • DaSHacka

      This is nonsensical, all our major adversaries are already probing and attacking our infrastructure 24/7.

  • dionian

    the author of this has been involved in other stories which didnt turn out to be true

Avicebron

It's clear, there is a screaming need for competent, well paid IT across all sectors.

  • gwerbin

    We used to have 18F (https://en.wikipedia.org/wiki/18F) but DOGE eliminated the office.

    • nomilk

      Not to defend the cut or weigh into politics, but how much of 18F was specifically security related? Because that's what's required here, rather than tech generalists.

      • sandeepkd

        I cannot comment on the specifics, they change with time. I had few interaction with technologists in one of the federal government body almost a decade ago. Those people were thoughtful, expert with first primitives and open to what you are doing as long as the intent is met.

                  Compare to that, most people in industry at this point are either business disguised as technologist or in some cases experts in one vertical.
      • saagarjha

        Bad software engineering practices makes security kind of hard to do

        • antonymoose

          18F was about digitizing public facing government functions and has nothing to do with the NSA.

          Aka bringing it up is flame bait circle jerking.

      • SoftTalker

        Not all outage are security incidents. Most are just technical fuckups.

      • riffic

        There is a well known intersection between reliability and security. Arguably they are two sides of the same coin:

        https://google.github.io/building-secure-and-reliable-system...

        GSA's 18F played a major role in applying Site Reliability Engineering (SRE) and modern DevOps practices to federal government infrastructure.

      • weard_beard

        In government security is political. And what we had with 18F was the beginning of a government office that was both tech savvy and politically savvy.

        Who decides when a DDOS of Trumpcare.gov that is traced back to Akamai servers was an attempt to extort government contracts or “those pesky Chinese state actors”?

      • deathanatos

        So, CISA, which … DOGE partially dismantled and has laid off about a third of its staff since?

    • gosub100

      Unless it has recently changed, all the smart people go to private industry, NSA employees are capped because of government payscales. You want highly skilled people? Fucking pay them.

    • biddit

      Lots of strong opinions on 18F's closure. Here is the GSA Inspector General's October 2024 assessment, done during Biden, and prior to Trump/DOGE actions.

      https://www.gsaig.gov/sites/default/files/management-challen...

      • rawgabbit

        I skimmed report but couldn’t understand what it said.

      • ajcp

        This assessment is about the larger Technology Transformation Service, which is very much still in existence.

        The only references to 18F are in a footnote citing two reports about 18F done in 2016 and 2017...so very much during Trump.

    • antisthenes

      Sure, that's a good example, but 250 employees is just a drop in the bucket across all North American IT departments.

      The problem is much more systemic than having 1 decent org.

      • datsci_est_2015

        Well, when you put it that way, it’s amazing how much just a drop in the bucket is able to accomplish. We should fill all of our buckets with at least a drop.

        • method_capital

          Yeah, the assumption that 18F would have prevented this is ... absurd. The problems we're facing are ai related, and external entities are more capabile, nimble, and motivated than our beloved government (right now anyway).

      • riedel

        You sure can hire a lot more external consultants to do the job now.

  • kart23

    this isn’t IT. it’s network engineering which is much harder to find truly good people

  • SlightlyLeftPad

    The trend is the opposite as the executives are cutting core infrastructure positions and replacing those with full stack software engineers at an ever growing, arguably unsustainable, cognitive load.

  • LastTrain

    Competent but understaffed is more likely the problem.

  • swozey

    Should we hire the million laid off engineers out there or go with the 22yo who hasn't done a bootcamp? We'll see!

    • cedws

      If you don’t hire juniors and grads you get the future you deserve.

      • orwin

        And sometimes you even get a junior who isn't just an AI parrot and your confidence in the future increases;

      • deathanatos

        But, perhaps, you could hire juniors or grads who like, have education in a relevant field, instead of someone who does not and goes by a moniker like "Big Balls"?

        • RaceWon

          I won't mention your moniker then..

          BTW: I actually am winning racer.

          • deathanatos

            Yes, because there is no difference between using a nickname on an Internet forum, and using it for your LinkedIn profile while you work for a corrupt government operation.

  • DotNetCorePowa

    Nnaaahhhh Ai will fix all of it. It's only been 1.5 years of vibe coding and everything seems broken beyond repair.

    But give it another year, trust me bro.

    • ceejayoz

      AI's being used to find good old human-coded vulnerabilities and rapidly exploiting them. Ten year old holes in PHP, for example.

    • drstewart

      > everything seems broken beyond repair.

      It does? Can you share an objective metrics that can highlight this that doesn't focus on single, isolated events?

      I know you'll provide this because of your strong anti-"trust me bro" stance and dedication to scientific fact.

      • sumeno

        ATC down, United down, Apple down, Chase down, Microsoft down, NSA down

        That's just going off the top of my head in the past 5 days

        • pferde

          Red Hat also had an outage of their customer support portal over the weekend, until late on Monday. Not as widely impactful as the others, but it was there.

        • drstewart

          The top of your head isn't statistics. I asked for statistics.

      • DotNetCorePowa

        Check the news lately... There is no needs to act like a "try-hard" with both your eyes closed.

  • jldugger

    Or at least an end to "ship it before thanksgiving break"

  • freecodeio

    I think most of "IT experts" are having a collective sigh right now and thinking everyone should vibe fixes into existence

    if only hunger wasn't an issue

    • toofy

      these “it experts” are basically used car salesman slimes who ignore their own orgs engineers.

  • okayman

    Guess who screams I’m competent the loudest? The incompetent workers.

    On a serious note, I see it as a paradox. Competency is built through struggles, and the non-competent do not want someone that seemed lesser at some point to outrank them, be the go to person.

ssalka

For me, 2 is coincidence, 3 is a pattern. Now we are at 4:

1. All flights grounded

2. Microsoft outage

3. Optimizely malware

4. Classified NSA network down

I don't know what, but something is clearly happening.

  • no-dr-onboard

    Very much agree with you that there is a pattern emerging. I think I agree with your conclusion as well.

    There are a couple of things to keep in mind with nsanet though:

    - It can handle anywhere between confidential to TS//SCI information

    - It is largely (completely?) an off-infra network, meaning that it doesn't run over the same infrastructure that the open internet uses. It uses an entirely(?) discrete backbone. SIPRNET uses similar architecture.

    - You cannot easily cross pollinate data from unclassified networks to nsanet. Everything from removable media to EMSEC is considered prohibited without proper channels being informed and paperwork being performed.

    - Lastly, we're unaware of the timing of this supposed source. This could have been over the last week or a month ago or <n>. Scant detail is given, so I'm unwilling to believe "lolz nsa pwnd" just yet.

    • jredwards

      I used to routinely watch people pull thumb drives out of SIPRNET machines and put them into NIPRNET machines and vice-versa.

      I would not be surprised if something similar was involved here. It worked with stuxnet, too.

      • no-dr-onboard

        So, that's illegal and against AUP.

        Beyond that it's also typically blocked by GPO prohibiting USB, so even if that were the case, the net result would be no real data transfer and a nice email to the DA mailgroup about the workstation and logged in user who triggered the alert.

        • sergent_moon

          I remember the cork board outside the dfac with phones and ipods screwed through the screen to the board... all the ones folks plugged into the sipr for one reason or another.

      • scottyah

        Stuxnet was preloaded thumb drives that were scattered in areas where people plugged them in on accident. Either you don't understand the DTA controls and processes, or the people around you were committing federal offenses and were being tracked.

        • mike_d

          Stuxnet was not randomly scattered USB drives.

          Unspecified computer parts were interdicted by US intelligence while being shipped to Neda Industrial and infected with Stuxnet. Four other external vendors working on the enrichment plant were also compromised, likely by Israeli intelligence physically entering buildings.

          Stuxnet then spread across each companies internal networks and onto USB drives that were then carried into the facility and plugged into the air gapped network.

    • ok_dad

      SIPRnet and the other confidential networks are transporting data on regular Internet but use encryption. It’s like a VPN.

    • tiahura

      Do you have an estimate of NSA uptime over the last 10 years?

  • riskassessment

    It occurs to me that even if the recent outages are mostly just growing pains from rapid/careless AI adoption, a sufficiently advanced AI cyberattack would be designed to be indistinguishable from normal outages.

    • ElProlactin

      > ...a sufficiently advanced AI cyberattack would be designed to be indistinguishable from normal outages.

      Aren't these being treated as "normal" outages by the media and powers that be?

      • TrueDuality

        They are so far, but I would expect the same kind of public do-not-fear response if this was a coordinated cyber attack and the US is at war with an adversary that has experienced the power and consequence of targeted cyber capabilities. If they didn't pursue their own program in earnest after StuxNet I would be shocked.

        We collectively are very much in arm-chair speculation at this point. If this is a coordinated cyber attack the US government right now wouldn't want to demonstrate weakness or fault. Iran would be the obvious culprit... but going full arm-chair on speculation... There was a major and disappointing result in our elections to the current powers that be...

        If this was a staged set of attacks carefully done to minimize casualties... I bet it would be enough to declare a state of emergency and get additional executive powers "legitimately". If that was the case I would expect more propaganda and less bury it under the rug if that was going on.

        Be careful out there the world is scary. Don't take any of this as fact, this is all personal speculation based on my own observations. Let me stand-up from my armchair and go for a walk...

        • ifwinterco

          The US has so many enemies both internal and external it could be basically anyone:

          - Epstein/SV network within the US flexing muscles in response to mid term losses

          - Iran, obviously

          - China, obviously

          - Russia, obviously

          - CIA attacking other parts of the deep state in some internecine struggle or other

          Literally all of those are completely plausible scenarios where the actors are motivated and capable

          • TrueDuality

            Absolutely agree. All of those are practical and viable threat actors.

            Based on my personal experience and to a certain extent sitting back down in my arm chair. I don't believe this is either China or Russia, they both have priorities that don't immediately involve the US and (as a government) the US is actively supporting both right now in very desirable (to them) ways.

            Internal US action and Iran both seem equally likely in my head right now, but this is purely speculation and attribution is the hardest part to actually nail down when a state actor is involved.

            This and the continued outages, such as the new Apple services outage, all collectively feel to close together without some evidence of a shared infrastructure causing it or a coordinated attack.

            I don't personally believe this is "AI agents hacking the planet". The companies involved (well at least half of the outages I've seen) have enough compliance and protection obligations they are regularly audited against that my Occam's Razor gut feeling tells me that this is either long established persistent and maintained access being detonated one-by-one OR its a focused attack by one or more teams under the same directive taking down significant US focused institutions for attack or reprisal.

            • ElProlactin

              > ...this is either long established persistent and maintained access being detonated one-by-one OR its a focused attack by one or more teams under the same directive taking down significant US focused institutions for attack or reprisal.

              Have you considered the possibility that this isn't even the detonation/attack? That this is testing and/or a modest demonstration of capability.

              If these are attacks, they don't at all feel like they're at the hands of people who have shot their wad.

          • ElProlactin

            It doesn't even have to be a state actor. Thanks to technology, even small groups can wreak havoc these days. And it's only going to get worse.

          • ambione

            how about israel?

            • ifwinterco

              The country that must not be named is always a suspect, but I kind of included it under Epstein/SV - ultimately that network all leads back to one place

          • weard_beard

            I would include in this list all of South America both government actors, disgruntled private citizens who feel they are beyond the reach of US authorities, and paramilitary or drug cartels.

    • xattt

      Iatrogenic outages would happen at similar times with overlap, versus convenient-for-narrative day-to-day outages.

      • mickdarling

        It's the fact the outages took the weekend off is kind of what is tingling my spidey sense.

        • GolfPopper

          Did the outages take the weekend off, or did the reporters and users of the impacted systems take the weekend off?

  • krauses

    The only problem with that bias is it only takes into account the hyper sensitivity and press these events are getting.

    While all flights grounded is significant on it's own, and would have got press, even in a world where the fear of AI or foreign interference wasn't so strong, but there's been more than a few substantial local and regional groundings due to software over the past 4-5 years.

    Microsoft outage is not new, Teams and Outlook have had more than a handful of outages way before the current doomer craze. While I had to search these for accurate recall, having worked closely with Microsoft for a decade plus, the recent outage is not isolated and to jump to conclusions is short sighted. For example, pre-doomsday summary of Microsoft's noteworthy outages: (1) Azure Outage -Sep 2018 South Central US (San Antonio DC); (2) Recurring multi-day Teams outage that effected users world wide February - April 2020 (3-4 outages in total); (3) December 2021 - Azure Active Directory outage locked people out of Azure and Outlook for several hours. (4) July 2022 - half-day outage for exchange, teams, sharepoint for most users. Same span outlook and exchange were down much like this last event for several hours. (5) Jan & Feb 2023 several outages for globoal Microsoft Outlook and network events.

    That's not even all of the events in the last 5-6 years.

    Optimizely, NSA network, and FAA mostly would have been under reported and or ignored by the general public.

    I'm not saying this time couldn't be different this time, but what is the thesis? AI, foreign actors, incompetence?

    "I don't know what, but something is clearly happening" is fear talking not rational thinking. Not all dots require connections. If you had full awareness of all the wild things that go on in one day, the amount of correlation you could draw from that if you're trying to tell a story would be pretty salacious.

    But correlation != ____________

    • omgwtfbyobbq

      Why would the FAA have been ignored by the general public? I feel like a 4+ hour stop on all planes in the US is going to be front page unless there's something bigger going on.

      I agree that correlation doesn't imply causation. At the same time, I think an increase in uncommon events over a relatively short period of time does increase the chance that they are correlated.

      Does it increase it enough to make it plausible or likely? No clue. But we've seen similar in Europe, and the consensus among intelligence/military analysts seems to be that Russia and Europe are in a non-military conflict of some sort.

      It might be just bad luck here, or the start of something similar.

  • BobaFloutist

    The devil is in answering the question "4 what?"

    Because then you have to define the "major" and "short" in "major outages in a short timespan", in such a way that

    A. Includes all four of those B. When applied to the past, would actually, meaningfully differ from the status quo (How often can something happen before it's alarming? Once a decade? Twice as often as that? Half as often?) C. Isn't a clear overfit, threading the needle to meet A and B, but is actually robust against perturbations.

    You will likely be right that this is a meaningful period, simply from the flights being grounded and the Microsoft outage, both meaningfully unusual events, but it's very easy to make a fairly challenging assessment seem simple if you just arbitrarily choose things to count on your fingers.

  • runjake

    Keep an open mind. We are pattern-matching beings.

    https://en.wikipedia.org/wiki/Apophenia

  • DSMan195276

    Would we have heard or cared about this one if the others hadn't already happened?

    I don't think it's a stretch to say that some number of services go down every day and we don't hear or care about them. Three or four services going down and/or being attacked over a few days is only a pattern if it's actually different from before vs. you're just more aware about them.

  • carimura

    US Banks down, United grounded a second time.

  • jjkaczor

    Heh - from "Goldfinger":

    'Mr Bond, they have a saying in Chicago:

    "Once is happenstance. Twice is coincidence. The third time it's enemy action."

    [From the original Ian Fleming novel]'

  • theanonymousone

    Train crash in Prague today: https://news.sky.com/story/prague-train-crash-leaves-46-peop...

    After the very deadly crash in Spain a few weeks ago...

    • IAmBroom

      AFAICT the Spain crash is due to horrendously irresponsible (non)implementation of train traffic safety systems.

      You don't need a spy if the target self-sabotages.

  • qwerasdf5

    Public companies have 4 days to disclose material cybersecurity incidents, per the SEC.

    While this rule doesn't apply to the NSA, speculation that recent down time from these U.S. companies being caused by state-sponsored cyber warfare seems unlikely.

  • LastTrain

    Four unrelated things?

    • causal

      Idk why you're being downvoted. Optimizely getting malware doesn't seem connected to Outlook going down last week in any meaningful way.

      It's easy to spot patterns when paranoid.

      • IAmBroom

        Because we don't know they're unrelated.

        If four politicians die on the same day of heart attacks, maybe they're unrelated. Maybe they're chemical assassinations.

        Same here.

        • causal

          4 systems going down in the span of 2 weeks for very different reasons is not atypical though. A more apt analogy would be "4 politicians died in the past week in different parts of the planet for different reasons". If headlines focused on those deaths it would also fuel conspiracy theories.

          > Because we don't know they're unrelated.

          They might be related! But the burden of proof is not on the null case.

        • LastTrain

          Thank you for explaining how the mind of paranoid conspiracy theorist works. For the rest of us it works like this: four unrelated events get treated as unrelated until there is convincing evidence showing otherwise.

  • irthomasthomas

    Wasn't there a bank outage too? OpenAi must be cooking. The new model is so strong it will mass destruct your ass!

  • xbmcuser

    They could just be asking AI to attack different things and find weaknesses as an attack everything goes down things go down 1 by 1 its more like testing. Or more cynical read it could be the plan to hose the american people for all the AI investment scare them into investing in and allowing ai data centers

  • jamal-kumar

    We're in an interesting time when Hanlon's razor seems to be being tested. "Never attribute to malice that which is adequately explained by stupidity".

    AI-accelerated attacks, I think most of the cybersecurity ones are attributable to that exploiting previous incompetences. But the flight grounding and microsoft outages? Looks more like direct incompetence to me

    Either way I think the takeaway is that we've been slacking for a long time and now we have something that doesn't need man-hours, caffeine, and obsessive human focus to take advantage of that fact. At least for those of us who have been in the field for a while there's lots of work cleaning up these messes for the foreseeable future but I been on calls with peers of mine in infosec who have definitely been losing valuable sleep and health over it

    • avidruntime

      Hanlon's razor reads more to me like a malicious thought terminating cliche than an actual law via averages that still requires testing. Anyone operating with Hanlon's razor can have their perception of anything easily hacked by simply pretending to be stupid, thus converting an act of malicious intent presumably brought about through some motive into an opaque act that requires no further scrutiny.

      • Abekkus

        Hanlon's razor is a useful tool for your personal human relationships, but it is a bad tool for geopolitics.

        • hyphenate

          burn after reading

        • IAmBroom

          It's a snark "law", not a real guidance policy.

          • avidruntime

            Agreed, although in common parlance the cliche has has developed more power than it was intended to possess, purely through habitual use. My main point is to point out that it does have a real chilling-like effect within public discourse. I see it most often in commentary about unfolding events. Inquiry into current events creates an impetus for further explanation from information dealers (governments, reporters, etc.) and when that inquiry happens out in the open, it benefits us in the moment and in the historical record. You only get so much attention from a reader and when the content is akin to eating your vegetables, readers look for excuses to clear their plate.

      • jamal-kumar

        Interesting perspective. Do you have any better proposals?

        • avidruntime

          Embrace the nuance of any given situation. Abandon the idea that there even exists a valid explanation in blaming cause on stupidity. The cliche doesn't convey information, its merely a characterization which gives permission to readers to stop further inquiry and our acceptance of this has been trained through basic mimetics. Find comfort in saying "I'm not sure what the attributable cause is" rather than deferring to a pop-sci over generalization of human nature. After that, its simply an effort of fact finding.

        • Timon3

          At the very least you should take "cui bono" into account, weighted by frequency.

        • folmar

          The reverse Hanlon's razor: malice by default, unless proven non-malevolent.

        • moralestapia

          What do you mean by "better proposals"?

          Of what?

          • jamal-kumar

            They have a really good point about strategic negligence and it's just the first I've ever heard of that angle

            • moralestapia

              Who is they?

              What are you talking about?

              The question I asked you is a specific one and it has a specific answer.

        • Detrytus

          Well, how about: "never attribute something that benefited your enemies to just a pure chance"?

    • floralhangnail

      At what point does widespread incompetence become malicious? AKA malicious incompetence

    • ramses0

      It seems we're in a 2-for-1 situation nowadays: "Hanlon's Cut-Throat"?

      • avidruntime

        Im more in the camp of just avoiding simple repeatable explanations for diverse, complicated, and nuanced events.

    • greyr0ck

      I would say the incompetence side also includes decades of 'get er done' attitude towards software design. We've been chasing fashionable frameworks and abstractions that may support velocity for developers, but not deep contemplation of the ramifications of design/architecture. The reward/risk balance was never in favor of slowing down to design secure software, it is in favor of meeting the complex and changing requirements of business as quickly as possible. Tech debt just got more expensive because 'poke it until it breaks' was always an option, but now we can ask an LLM and it can work tirelessly in the same problem space. Time for everyone to grind fundamental CS concepts, and aim to reduce complexity. On the bright side an LLM can help with that as well, just remember to append "make no mistakes" to the end of every prompt!

  • lh712

    Although I don't know the answer, and have not done the work to analyze this situation properly, in cases like this I find it helpful to try to focus primarily on the overall epistemological framework rather than only the matter at hand (since it is easy to get lost in all the uncertainty, lack of knowledge, and presence of disinformation); an especially useful and simple tool here is Bayesian inference.

    For instance, I have heard that Iranians have developed rather sophisticated cyber-warfare capabilities. I don't know if that is true, and if true, I don't know how likely it would be that they would have reached capabilities to cause the observed outages. But even if you assume that the probability (or rather degree of belief) is quite small but still realistic, say between 0.1 and 10 percents, than considering the current situation it might be absolutely rational to see it likely that they are behind these outages. The full analysis requires estimation of the probability that this is all just a coincidence, and of other major "non-coincidental" branches. But one can do this at various levels of precision and explicitly observe how the output depends on the priors and the estimated probabilities (and therefore, on the assumptions).

    • ted_bunny

      As always: how does Bayes' Theorem and assigning arbitrary percentages shed any light on this topic? If you took any mention of Bayesian inference out of your comment, would it lack any content or insight? I have encountered the concept for years and it just seems to make explicit what everyone already does implicitly. Not that there's no value to that, but I think it's very rare that it's worth invoking.

      • lh712

        It is hard to give an answer to a comment like yours since I have no idea at what level are we discussing the topic.

        First of all, I have not said that just assigning arbitrary probabilities sheds much light on the matter. Rather, what I alluded to, is that even rough estimates might often shed a lot of light on the topic. The Bayesian formulation helps to separate the problem into relevant parts, make the relative importance and interdependence of various factors that enter the reasoning clearer, and makes the assumptions more explicit. But in my post I have not really explained anything; I have just pointed to a tool somebody interested might use to critically assess similar issues. So yes, if you took it away, the post would lack an important part of its message, namely a pointer to a framework that the OP (or anyone interested) might find useful. However, I do not plan to give a treatise on Bayesian analysis in a HN post.

        As for the other angles of your post: 1. Yes, the usage of the framework absolutely can shed light on many matters of this kind. What is your background? How much experience do you have with statistics or inference in general? There are many good sources where you can learn about these methods. A somewhat controversial one, but freely accessible, is E.T. Jaynes' Probability Theory, which actually directly addresses some aspects of what I spoke about. 2."it just seems to make explicit what everyone already does implicitly" People apply very bad judgement to matters like these all the time, this forum included. And even if people all used those things implicitly (which often they don't) it would still be worthwhile since it allows one to inspect the argument and find flaws or observe what are the dependencies on various assumptions.

  • rpdillon

    Apple's services are down as well.

  • Betelbuddy

    What kind of CLASSIFIED network is this one, that when it goes down we hear about it on CNN?

    • scottyah

      I assume most trusted news networks would have some staff that are cleared so they can read a bit more detailed news, then help frame the messages that go out to the masses.

      • ted_bunny

        They have intelligence mouthpiece contacts whom they quote uncritically, because they have no other option. That becomes the new truth through Citogenesis.

      • IAmBroom

        That's a silly assumption.

        News professionals would never be granted "need to know", and therefore would never be granted any level of secure access at all.

    • layer8

      Classified ads aren’t a secret as well.

    • walrus01

      The size and scale of something like SIPR, JWICS mean they're interacted with by tens of thousands of people at facilities worldwide on a daily basis as part of their regular relatively mundane work. It would be hard not to notice.

    • joshymochy666

      This! ⬆

    • lukas099

      The existence and status of the network are not classified, just some of the data on it, I'm guessing.

    • ecshafer

      The kind that you need clearance and polygraph to get on.

      • Betelbuddy

        Here is a question for your next polygraph test: "Are you the source who told CNN about our internal IT stuff?"

  • holgerschurig

    Yes, you don't know. And neither do I.

    What I however know is that somehow the human brain likes pattern matching. We look into the sky and see patterns made by the clouds. We see some random cloud formation, match it against 100000 patterns and say "This is a car". We look at ink blots on a paper, match it and say "This is a knife" (Rorschach test).

    Nothing of this is true ... it's just that we matched a pattern. And that he hold something plausible.

    But ... thinking something is plausible is far from being it true. To draw an actual conclusion, we need a rigorous thinking process. Based on logic, not assumption. With facts, not assumptions.

    If I would have to describe the current times (e.g. the last 15 times) then it would be the sudden lack of all this virtues in people engaging with other people. We seem to be certainly back in the old times, where rumors are hold to be true ("This red haired lady with a dark cat is certainly a witch. Burn her!"). And, worse, often people base actions on this hearsay, so we are also a bit back in the Lynch justice mindset.

    > I don't know what, but something is clearly happening.

    And I know, something IS happening. That's an inherent truism. Example: You, dear reader, are breathing. Breathing is "something". It is happening. Clearly even. I mean, if we start to write extremely vague, then another human matching pattern can see anything in it. We shouldn't talk in horoscope language and we shouldn't assume that horoscope language contains anything of value.

  • saagarjha

    The pattern is that a bunch of software kind of sucks

  • hrncn

    I am just surprised this is not happening all the time, after piling on layers upon layers of software and merely looking for a checkbox to check and someone else to blame (and pay). Even the vertically integrated tech companies that used to be in control of the whole stack are having their Outlook moments now, everyone else is worse off.

  • scottyah

    Everyone had to jump off VMware because the price got jacked up, and the people who were forced to rebuild the systems in a hurry weren't the ones who architected them. I'll bet there is a big learning curve for these systems that were slowly optimized over time, but now people are switching to Openshift and whatnot where those optimizations and Lessons Learned are mostly thrown out.

    AI cyberattacks, foreign interference, and Ghost in the Shell type stuff sounds cooler though.

  • AstroNutt

    You forgot about the Chase bank incident yesterday. Supposedly it was a software glitch. So that makes 5.

  • BLKNSLVR

    Timed to peak with GTA 6 release and Anthropic IPO.

  • gmerc

    DOGED

  • vrganj

    The deadline for Genesis Mission [0], the US gov project to integrate its systems as well as private systems into a centralized AI platform to deliver its first report is at the end of November.

    I saw some speculation here [1] that it might be the root cause.

    [0] https://en.wikipedia.org/wiki/Genesis_Mission

    [1] https://substack.com/@thedreydossier/note/c-389611426

  • camillomiller

    Add 5, apple services down

  • moralestapia

    PRISM update is being rolled out.

    It's a big change so hiccups are expected.

  • jmyeet

    In the early days of the Russian invasion of Ukraine, Russia faced a big problem: it had hardly any of the tanks it paid for. Why? Corruption. Basically it worked like this: a general would get money from the Kremlin to design and build new tanks. They'd build a few but for whatever reason they'd be found wanting. So a new contract was written up. But what happened to the money? It got pocketed by the generals, the contractor and a bunch of other people in between.

    That's basically what's happening to the US right now. It is corruption from the Oval Office all the way down. Pardons are being openly sold. The news was manipulated ("a ceasefire is imminent") to profit off the oil markets. Stock trades on insider information abound. All of this is an inevitable consequence from a Supreme Court who manufactured out of thin air substantial criminal immunity for the president. This built on other decisions where this Court in particular just could not see corruption (eg Snyder v. United States [1]).

    In the private sector literally nobody is ever punished for cost-cutting on systems to the point that they break. In fact, they're rewarded for it. It's entshittification from the CEOs all the way down.

    We'll probably find as more of these incidents happen that AI code was written and reviewed by another AI, never tested and deployed to production. Foreign actors will be blamed, Congress will be blamed, etc. As an aside, the DoT is asking for $50 billion after the FAA outage (the worst in 25 years). Then Representative Mullin voted against $25 billion for the FAA in the Infrastructure Investment and Jobs Act in 2021 [2].

    Now the administration wants to sell this narrative that they've been hacked. No, they're just corrupt and everything sucks as an inevitable result. Everything is shit. Nobody is hacking us.

    I honestly don't know how we get out of this but we need a hard reset of some sort. One such hard reset was the French Revolution. There are an awful lot of people who need to spend the rest of their lives in jail, penniless.

    [1]: https://us.transparency.org/news/supreme-court-undercuts-fed...

    [2]: https://heritageaction.com/scorecard/members/M001190/117

  • ck2

    the pattern is more and more poorly trained people are effing around with running "AI" agents on their massive, fragile codebases

    none of these systems are even vaguely similar

  • throwaway5752

    "Mr. Bond, they have a saying in Chicago: Once is happenstance. Twice is coincidence. The third time it's enemy action."

  • hebelehubele

    Pretense for Iran ground operation

    • SamBam

      Iran's strong hits against our military, which were barely reported, could easily have been the excuse of they wanted to go that route.

      They don't. This administration thought they could Venezuela Iran from the air.

    • everdrive

      Yes, there are no legitimate actions in the world. Everything which seems to happen is actually just a pretense so we can justify our actions.

      Even more importantly, when these pretextual actions occur, everyone interprets them the same way and agrees that the secretly-planned action is now the one we must take.

    • swat535

      Do they even need a pretense these days?

      It has come to my attention that the political elites and the Epstein class simply do as they please with virtually no consequences.

noah_helios

Thinking a few layers deeper: They must have a strategic reason for so loudly telling everyone the network is down. I would assume that, normally, these kinds of IT disasters would be kept very quiet.

So, the real question, is: "why are they making this public?"

sourdecor

NSA to cut up to 2,000 civilian roles (2025) [0].

[0]: https://thehill.com/policy/defense/5286561-nsa-cut-up-to-200...

MomsAVoxell

Considering the heinous violations of human rights at massive global scale that this service commits, this is finally some good news.

nektro

the statement: https://www.nsa.gov/Press-Room/Press-Releases-Statements/Pre...

pjdesno

FWIW, I had a summer internship 40 years at BBN, working on the obsolete and soon-to-be-retired switches running the ARPAnet; one of their other customers was the NSA.

I'm pretty sure they were calling that network the NSANet back then; I assume it's gone through many generations of upgrade since then. I don't remember much about them as a customer, except that (a) we weren't supposed to mention their name, and (b) they were running a horribly old version of the switch firmware. My understanding was that they literally tested stuff for years before deploying it.

It was a long time ago, and I was just a summer hire, so I could have this wrong; in addition things may have changed in the intervening 40 years.

  • jiggawatts

    I’ve been peripherally involved in builds of “very high security private networks” for government.

    It’s ass backwards.

    The entire process is about compliance, the actual security is totally ignored.

    One vendor I spoke to was scouring eBay for old Juniper VPN appliances that still had some ancient firmware that could no longer be “downgraded to” on new ones.

    Notably, this old firmware included the back door used by the Chinese government that was trivially exploitable! [1]

    Why that version!?

    “Because that specific point release was certified years ago, and they never updated the certification subsequently.” was the almost verbatim explanation!

    It’s insane, but that’s how government drones think. You can’t snap them out of it, they’re unable to comprehend any other way of doing things.

    Details like “security is a treadmill, you have to keep moving” just doesn’t compute.

    [1] https://en.wikipedia.org/wiki/Dual_EC_DRBG

  • acquacow

    Odds are they are still using it and they are still on the old switch firmware =)

btreesOfSpring

I feel like something started after Elon purchased twitter and made all those personnel deep cuts. In the era prior, it wasn't too hard to look around and see people who were dragging productivity down. Their cuts would be an improvement. Twitter into X looked to cut to the bone and despite a few hiccups, nothing catastrophic happened to the service layer. This ushered in a new era of tech department downsizing that feels like it is still happening today. Ask any new graduate how the job search if going. Sprinkle in a little DOGE + time and we are starting to see the exhausted people supporting these glued together systems experience real large scale failures.

Yeah, you can do fine with only one set of eyes on a problem. Yeah, it is more "efficient" in the short term. Culling lots of do-nothing people in your organization isn't going to produce many tears. Even so, devs and IT departments are now missing all sorts of backup and vital personnel that keep the gears oiled and running.

  • abustamam

    I have a friend who works as a process engineer at Tesla and she and her team spent weeks working on a project to save $10m on the fabrication process of Model Y/3, and this was in the shadow of a few rounds of layoffs (details are fuzzy since it was some time ago).

    Then the board voted a $50b pay package for Elon Musk and she's like... "wait, why the hell are we trying to save $10m when we could save 50 times as much and not even be at 1% of his pay package?" I have no answer to that.

    Also to your point, the value of someone's work isn't always tangible or directly correlated to the work they do day to day. I sometimes have some extra slack/bandwidth at my job but it allows me to jump on any incidents or anything that requires urgency faster than anyone else.

    • rebuilder

      Workers create wealth for owners to extract. Your friend was creating so Musk could extract.

      • andsoitis

        That worker is way more replaceable than Musk.

        • nign

          In a sense, yes of course. That has zero relevance to the comment you're replying too though. How much a person is paid for the value they produce does not necessarily correspond to how easy it might be to replace them.

        • gpvos

          Very debatable.

        • idle_zealot

          Musk doesn't even need a replacement.

        • mschuster91

          Oh hell no. You could probably go and do a Luigi on most F500 CEOs and nothing too bad would happen (other than the stonk markets taking a nosedive out of panic).

          But try to replace the actually vital people? Oh boy. SpaceX for example, if something were to happen to Musk the engineers would probably be quite happy of being rid of the constant noise. But if something were to happen to Gwynne Shotwell who is the one running SpaceX in practice? Whoops. No bueno.

      • tosapple

        and that's why we opened up to China and created the WTO.

        first we extract gdp then we extract jobs then we extract technology

        we're getting a good hard look at what 'deregulation' looks like currently.

      • refurb

        You’re about 35 years after the collapse of communism so please bone up on your economic theories

    • refurb

      Some context on the $50B pay package is in order!

      The original package was approved in 2018. Note the $50B was if Tesla hit certain valuations, which at the time nobody believed.

      A shareholder lawsuit and left wing judge decided to invalidate it in 2024, after Tesla hit those valuations.

      So shareholders voted again to give Elon the pay package, and the judge rejected it.

      Then the Delaware Supreme Court overruled and Elon got his $50B.

      I’d say the situation is more like getting equity at a startup that is worth nothing, then it turns into billions but you lose it and the board gives it back.

    • derangedHorse

      Proportionally, it costs close to nothing to make new shares for Elon's pay package. Since Tesla doesn't have any treasury stock, it also won't have the indirect cost of dilution bringing down the balance sheet.

      Rejecting the pay package therefore wouldn't mean "saving" X amount of funds.

      • whall6

        Well, no, because there’s opportunity cost involved here. $50 Bn of Tesla stock is roughly $50 Bn of cash. You could have paid it out to employees. Or paid it out to other execs. Or anyone.

    • bongofizzbuzz

      but Elon is not using that money to buy a yacht. he will put that money back into future technologies. Elon is really different from other billionaires.

    • jlarocco

      Well if it weren't for her hard work Musk may have only made $49.99b.

    • raverbashing

      > .. "wait, why the hell are we trying to save $10m when we could save 50 times as much and not even be at 1% of his pay package?" I have no answer to that

      It's funny how the smartest technical people keep falling for this "Management 101" trick. The reality distortion field is strong here

    • stackghost

      >Then the board voted a $50b pay package for Elon Musk and she's like... "wait, why the hell are we trying to save $10m when we could save 50 times as much and not even be at 1% of his pay package?" I have no answer to that.

      The answer is the very next paragraph of your comment!

      >Also to your point, the value of someone's work isn't always tangible or directly correlated to the work they do day to day.

      Elon is negative value and yet gets paid a stupid amount. Your friend is saving the business millions and doesn't see a dime of that.

      Welcome to late-stage capitalism.

      • wallst07

        This is all purely subjective.

        The friend can easily quit and find a better job paying more, why don't they? Likewise, Elon could step down for whatever reason and we'd see what happens to the stock price, maybe it goes down 50b, maybe up, who knows.

        I do know that Elon leaving will have a much larger impact (one way or another) than 1 process engineer.

        • lossyalgo

          > The friend can easily quit and find a better job paying more

          How can you assume that to be true? It depends where they worked and if there are similar open jobs nearby. Are those skills specific to Tesla production line or transferable? If they need to move to find a new job, do they have kids that also need to be moved? There are so many factors involved.

          Also who says 1 process engineer? We're talking about a $50B pay package to a guy who already has nearly infinite money - that's enough money for hundreds of employees.

          • robin_reala

            Let’s say that by “hundreds” you’re assuming 500 employees. If you split $50B evenly between 500 employees each employee would walk away with a pay package of $100M. You could afford to pay 100,000 employees a pay package of $500K for one Elon’s salary.

      • BLKNSLVR

        New word: capitulism

        A capitalist society's inevitable capitulation to the lure of capital over all else. The moment that the final nail is hammered into the coffin of what used to be known as the idea of the 'free market'.

    • wallst07

      Ok, what is your point? Your friend makes more than other people that work there are making. Should everyone be paid the same, who gets to decide what multiple?

      None of this objective or fair.

      • anon7000

        Um, the point is why even care whatsoever about cost cutting to potentially make the product worse, when you could just cut CEO pay? The CEO pay is at a point where it doesn’t even change their life meaningfully in any way.

        Yeah it’s not objective or fair, that’s what we’re pointing out. That makes it objectively bad, inefficient, and dare I say fucking stupid

        • somenameforme

          He's paid in newly created shares which have literally zero cost for the company outside of reducing the value of shares, but since their own holdings are negligible it's still a cost of approximately $0.

          • hvb2

            You can spend money only once? As simple as that.

            For a start, the opportunity cost of that sum is non zero

            • somenameforme

              They're not spending money. They are giving him newly issued shares. In this exact scenario it's even a bit more nuanced because his package was in the form of options. So he 'bought' the options meaning Tesla gave him a bunch of newly created stock and then kept a chunk for themselves as well as the cost of the options.

              And all of this was also contingent on him dramatically increasing the value of the company. If he failed to do that, he would have gotten $0. It'd be akin to in this scenario where somebody's friend was working on a $10 million saving if her entire team received a salary of $0, and contingent upon the completion of the project, they'd each have received a fraction of the savings generated by it. If they failed, or it somehow did not achieve the desired savings, they'd receive $0.

              And furthermore payment would also be in stock meaning that their compensation would be subject to the success of the company, and also would be worth less in "reality" than it was on paper since selling would reduce the price of said stock - probably negligibly for a relatively small amount, but it's still a nuance that's important. If Musk tried to convert his trillion (or whatever) to cash, it'd end up being worth much less than its nominal value.

          • wallst07

            Bingo, these people don't understand.

            It's all CEO bad tax the rich BS. I wish I got paid more sure, but I'm not pretend that I'm more valuable to the company I work at than the CEO.

            If they made me CEO and I implemented a pay increase package and decreased my own, I'm pretty sure the stock would tank by more than the amount of the pay increase for everyone else.

            • komali2

              > I wish I got paid more sure, but I'm not pretend that I'm more valuable to the company I work at than the CEO.

              I genuinely want to understand this position: don't you feel a little gross denigrating yourself like this? To purposefully undervalue yourself compared to another person?

              • rightnutwingjob

                > to the company

                > another person

                Do you see how these aren’t the same thing?

                • komali2

                  Both the company and society use the same scalar to measure the value of a person and indicate their worth. There is no difference between "value to society" and "value to a company" under this system.

              • transcriptase

                Some people being objectively more valuable to others and society is just reality. It's measurable. Shaming someone for not partaking in some idealistic fantasy only suitable for a preschool classroom or resentful subreddit is the bizarre position to take.

            • DANmode

              Privately-held companies who perform stabilizing actions like this don’t have an outside sentiment or stock price to sweat.

            • elar_verole

              > but I'm not pretend that I'm more valuable to the company I work at than the CEO.

              I mean even if that's true and you believe that, are you like 250000x less valuable than your CEO ?

  • notesinthefield

    I shed no tears pointing out this is happening all over US gov infosec. We’re all teetering on “let it burn” ands barely keeping it afloat. Awful lot of utterly clueless, paper tigers defended by the fact simply no one else wants to do these jobs for gov. I’m considering draining savings and taking a break for the first time since I was 14.

    • stevarino

      Was let go in June. I haven't had the interest to jobseek, and will be fine for a while longer.

      Books, videogames, travel, photography, civic engagement, coffee shops, and just being a bum is pretty awesome. I think the Chinese have been calling it lying down?

      I wish I was rich enough to continue this lifestyle. Inverse FIRE, where I scrimp and live miserly, riding on just-okay savings?

    • notyourwork

      Big tech swallows the talent because we refuse to pay government employees commensurate with the scope of responsibility. It’s sad and a slow drain.

      • lanstin

        Government isn’t just low pay it’s also things like having to start at 8 am and wear nice clothes and do water fall with so many planning meetings. I worked at AOL for a long time and looked hard for government jobs to return to paying work from a few years of stay at home parenting. There just was nothing about creative technology culture that could exist in DC based US government. One hopes it is better now. I am happy to be paid a lot, but even more happy to be able to work during the hours that I am most productive and stay up late when the muse strikes and then sleep later the next day. And having worked my way up to software, there is no way I would be able to wear the tan pants and dress shirts and ties I had as an admin person in DC.

        • abustamam

          I don't mind wearing nice clothes if I feel like it but if literally no one else is gonna see me but my coworkers who also wish they could wear jeans then maybe the dress code is just stupid anyway. The concept of "professionalism" (and also the concept of a 9-5 and a lot of arbitrary concepts that made sense 70 years ago when there needed to be a distinction between blue collar and white collar workers but I digress) is just silly.

          I do great work remotely and I just wear my pajamas all day, and work sometimes at 7am, sometimes at 7pm, sometimes at 10pm (honestly depends on my token usage). I'm comfortable and I'm effective, what else could you ask for?

          There are many folks way more talented than me who would never work a government job because of silly reasons like that. And it's like... Why?

          • appreciatorBus

            I disagree.

            Dressing up every day makes you feel great. I recommend it to everyone.

            I had a job where dress clothes were expected and I miss it dearly.

            • hilariously

              Nobody is stopping you from dressing up, we're just mad you want to force your opinion on everyone else. I had a friend who wore a suit every day to his casual workplace and looked like a boss, still didn't want to do it myself.

              • lossyalgo

                I've been at jobs that made me dress up every day. For years. Never again. It's a horrible waste of time and achieves nothing.

        • corysama

          Make Service Sexy Again!

      • gedy

        It would be fine if housing was cheaper

  • MrBuddyCasino

    What happen right now is that every organization on earth is under attack from automated LLM bots. This is why we are currently seeing so many outages. Nobody admits that this is the case.

    • alchemist1e9

      Everyone is assuming it’s LLM bots but it apparently isn’t.

      https://www.washingtonsun.com/technology/bank-ceos-nsa-tell-...

      > Bank CEOs, NSA Tell Congress That AI-Driven Patches Are Behind Outages

      > Artificial intelligence is not causing the disruptions but identifying gaps that are difficult to fix.

      • theptip

        It’s an interesting angle, but this is basically the same cause as GP.

        The base threat level has increased dramatically. Now everyone is racing to use Glasswing/Daybreak/whatever to fix the bugs before someone external finds them.

        So yeah, maybe not a direct result of bots attacking, but still an indirect one.

        • autoexec

          I think there's a big difference between malicious actors using AI bots to break everything and multi-billion dollar corporations and governments vibe coding patches that break everything. You can't do much to stop AI bots from testing your defenses, but pushing a bunch of untested updates to everything in production in the middle of a busy day is a choice.

          These kinds of outages will just point out which idiots have no business managing these systems in the first place and they'll hopefully all be fired so they can be replaced by people who know better.

          • balex

            This isn't a solved issue. People don't know better yet. That's why there's so many startups in defensive AI.

    • anon373839

      I am very reluctant to buy into that narrative because it's just a bit too convenient for a certain AI lab that desperately needs a strong IPO and has no path to justify its valuation if it can't find a way to monopolize the technology.

    • brookst

      Who do you expect to “admit” this? It’s common knowledge. Everyone on HN knows it. It’s like “admitting” that the sun rises in the east.

  • jart

    Those do nothing people were definitely doing something. Now that the system has shaken the beehive, it got to find out what that is. Resets like this are a natural thing. If you don't understand how a system works, you break it, and have fun building something better.

    • libertine

      The thing is that those resets don't affect much billionaires, but more so regular people - you know those that have wait in lines, that an unexpected expense could throw them off, an accident could be life changing, etc.

      There's nothing natural about a billionaire with hints of racial supremacy coming in to cut costs on a complex system of a state.

      State institutions aren't businesses.

      • jart

        I'm a human supremacist. Right now those billionaires are flooding your online communities with robots, just like how you flooded working class communities with foreign ethnic groups. Soon I will be a grey goo supremacist. Then a paper clip supremacist.

  • bamboozled

    It is a crisis, look out the list of high profile outages this week and it's pretty damming, I think you're right, downsizing combined with "AI" has not been a great combination.

    The thing is, do people actually care enough to make the required changes to improve, or is this it ?

  • insane_dreamer

    Nothing critical is dependent on Twitter (which has had plenty of hiccups), so what "worked" there doesn't "work" on critical systems

    • btreesOfSpring

      That is the point of the post. It started with something very public and non-critical and has slowly made its way through the full pipeline into real systems our infrastructure relies upon. The consequences are finally showing. It is too bad most of the people who were raising the red flags were sacrificed to layoffs & dismissals.

    • dathinab

      this is a bit misleading

      Twitter was so insanely resilient that in many places it was more reliable then any "official"/"emergency" communication channels for info about natural catastrophes and similar

      so people did rely on it for "critical" thing

      and that was gone with Musks changes

      which is one of the things people tended to overlook there was a huge decrease in reliability/resilance after the huge cuts, it's just that the final outcome was still "just fine" for what most people in the US/EU used it for

      • nomel

        > and that was gone with Musks changes

        Is there data/site to compare uptime before and after?

    • TZubiri

      Sometimes the industries were the stakes are lower overcompensate and end up having tremendous quality and work ethic, in video game design for example crunch and 70 hour workweeks are a thing, while established physicians can do with 10-20 hours.

      • bigfishrunning

        Crunch and 70 hour workweeks are a symptom of poor management, poor planning, and a desire to wring the workforce dry, not an indicator of tremendous quality and work ethic.

    • esalman

      The same person who made cuts in Twitter also made cuts in US federal government.

  • pokstad

    Or maybe it’s the enshitification from the AI slop movement. People who don’t understand the ramifications of their PRs making giant frequent changes to codebases while maintainers feel pressure to hit the approve button.

    • nukanse

      No. It's war. Just yesterday the french president acused Iran of a flood of attacks, and Newsweek titled "Cyber Attack Hits 178 Major Websites".

      • tru3_power

        So you think all these outages are part of this? Could you expand on why you believe so?

        • phoghed

          No more than the person confidently claiming AI is the cause most likely.

          • King-Aaron

            This is frustrating me immensely with the whole conversation right now. Not one single person in comments land knows exactly what is going on. So the passive agressive "oh why don't you explain" kind of comments are just so inane.

            • Paracompact

              > So the passive agressive "oh why don't you explain" kind of comments are just so inane.

              I see nothing passive aggressive in tru3_power's message, if that's what you're referring to. What's the alternative?

            • joquarky

              As someone now in their 50s I am embarrassed every time I see this now.

    • alfalfasprout

      The two things are related. Rampant fixation on costs while not wanting to reduce velocity means cutting staff, outsourcing w/ low skilled labor, and quadrupling down on AI for everything.

      We shouldn't be surprised at where that's led.

    • thephyber

      I don't think the NSA's high side system uses almost AI for coding. I know someone who works for the government contractor that works on one of these systems. They are stuck using old technology (5-10 years old) and most of their time is spent either fighting to get "urgent" patches tested and approved or trying to keep up with requirements changes.

      That said, who knows how bad things have gotten in 2025+. The Trump political appointees seem to be largely incompetent and sycophants never improve the resiliency of systems.

    • btreesOfSpring

      This is another factor that all started with the twitter purchase and made its way into the fewer people is better movement in tech.

    • gilfaethwy

      Let's not judge people submitting AI slop PRs too harshly. Maybe AI slop against open-source codebases, but a lot of the proprietary AI slop is a result of management directives and organizational pressures, not engineer desire to merge in crap code.

      This is doubly true at this time of year, when we're likely to see people pushing to get work delivered by the end of December so that they can include it in yearly reviews. I'll happily merge PRs I don't understand if that's what keeps my job.

      • vrganj

        > Maybe AI slop against open-source codebases, but a lot of the proprietary AI slop is a result of management directives and organizational pressures, not engineer desire to merge in crap code.

        Then it is a failure of software dev individualism. There is a long-proven, battle-tested solution to management making unreasonable demands from labor: Unionization.

  • mschuster91

    > Twitter into X looked to cut to the bone and despite a few hiccups, nothing catastrophic happened to the service layer.

    That is because Twitter was and still is an incredibly well built system. It took a while and a lot of fail whales [1] until it got to that point, though.

    But ever since that, there hasn't been that much innovation on the platform. Sure, we got native link shorteners, native photo hosting, native video hosting, private messages and circles. And now the CSAM generator also known as Grok. But... neither of these touches the "core" of Twitter, which is a system to fan out short pieces of text extremely fast across up to 250-ish million followers (Musk) while respecting stuff like account blocks, it's all bolted on top of that highly reliable infrastructure.

    The danger is now that, say, you're Microsoft... you think you can cut costs by cutting off the greybeards. It worked at Twitter after all. But unfortunately, by all accounts I've come across, Azure is a brittle-as-fuck shitshow often requiring manual action by Microsoft employees, with cost-cutting already going as far as to offshore support for US Government Cloud customers to China up to 2025 [2].

    Just because it worked on Twitter once, it doesn't mean it can even work at one other large company. But unfortunately, CEOs are vulnerable to herd mentality, sheeple following one leader mindlessly.

    [1] https://knowyourmeme.com/memes/fail-whale

    [2] https://www.cnbc.com/2025/07/18/microsoft-china-digital-esco...

lucascdotnet

I wonder (and have no idea) whether the increased reports of services being down is a case of the Baader–Meinhof phenomenon, or a actual increase in failure rate.

  • btown

    If I was asked to armchair-pattern-match, I'd say that there's a combination of two things: (a) a wider fear-based environment where rapid modernization is being pushed heavily, and (b) an increase in tools that give false confidence on change management.

    AI is tuned to speak confidently and ship independently; it is rare that it will suggest that one go back to a broad survey of stakeholders to ensure no Chesterton's Fence is being torn down (https://fs.blog/chestertons-fence/). "I've read the documentation and logs, and nothing suggests this system needs these legacy capabilities/components/capacity" is something we hear all day, and it is a siren's song if there ever was one.

    Couple that with a general (and valid!) consensus that modernization of legacy systems must be accelerated for a variety of reasons - to mitigate this new and continuing "Black October" of hyper-agile agent-driven cyberattacks (whether the network is public or private); to have more flexible infrastructure for new features to stay ahead; to allow human teams to show their efficacy in a world where technology jobs are increasingly at risk - and you have a perfect storm where change management is no longer what it once was.

    • clickety_clack

      10 years ago, most work happened in Word or Excel on a desktop app on your computer, so you weren’t depending on a constellation of cloud based SaaS vendors. If outlook went down for an hour, you’d just continue working without the disruption of an inbox. Now, the path to do your work requires online vendors to be online, so if they go down you’re blocked.

      • KellyCriterion

        This ++1

        Some weeks ago, our Claude subscription failed - most people stopped working and switched to tasks they had postponed.

    • joe_the_user

      I don't necessarily agree about the warnings. AI often gives me "now be careful" warnings about it's output along with the caveat at the bottom of each ChatGpt page.

      I suspect the situation is that AI's caveats are just easier to ignore than a person's warnings. Plus all the other factors you list. Also, AI tends to give a person the kind of communication they ask for. I always ask for options but I suspect other people ask for just answer and so they get them.

      • troupo

        > AI often gives me "now be careful" warnings about it's output along with the caveat at the bottom of each ChatGpt page.

        Doesn't stop so many people from believing it wholesale, and doesn't prevent so many programmers from saying "I don't review code anymore"

        • icantevenhold

          To be fair many also didn’t really review the code before LLM tools and just lgtm.

          Also the amount of code that is being produced (at least in some places) is almost unreviewable

      • causal

        Yeah I think most people just prompt AI to close gaps until it gives a green light to go ahead.

        Asking the coin-tossing machine if it's safe until it says it's safe. :)

    • cyanydeez

      I think he's just pointing out that AI has made news of IT failures "more interesting" and thus, more reports are getting out where in the past, they'd not be interesting.

      I think you went off somewhere else.

      • joe_the_user

        The OP is giving an alternate explanation of the general phenomena that this headline is a part of. Nothing wrong with describing the framework that software "drift" is happening in.

        Which is to say, a parent post here on HN doesn't narrow the topic to it's focus. That's what makes discussion here interesting.

    • psunavy03

      Modernization of legacy systems also "must be accelerated" so executives can lay off everyone maintaining the legacy systems; let's be honest here.

      • 6510

        That legacy system that ran out of bugs to fix half a decade ago. I'm not even sure if it still counts as maintenance but if anything comes up it will have my full attention.

        There was this tv program in the 90's about a fully automated japanese warehouse. The staff was one maintenance guy. They ask him if it was a lot of work. He said, everything is new then he made the face like a puppy to fill an awkward silence.

        I've often joked about the mechatronics fortune in our train yard, they need to study hard, get the diplomas, get experience. Then the only tool they get is a flashlight and they have to walk around indefinitely not finding anything. Hundreds of employees like that. A good day is when they all find nothing, there are many good days, they are not suppose to find anything. If anyone does find something it most probably wont be you. They apparently still find enough things to prevent the full switch to AI and those pretty line scan pictures.

        https://upload.wikimedia.org/wikipedia/commons/7/74/Line_sca...

        The new (modernized) trains have IT guys, they usually have red/purple faces and run up and down the train with their laptop. They plug it in all over and the color gets brighter. Bright enough that I don't make jokes about it.

  • Someone1234

    That's my impression; a total of zero of the popular recent ones have been related to "AI" or Cyber attacks, but every comment in every thread, talks about those two as if they were the underlying cause.

    The FAA's recent outage for example, at its core, seems like people trying to rush needed upgrades that were slow for a specific reason (e.g. reducing the testing window for 90 to 30-days).

    The Microsoft outage seems like three different things linked together (and poor "full failure" recovery prep, lengthening the outage).

    The bank "outage" may not have even existed, 2/3 of the banks claimed they didn't have an outage at all, and the other said it was a limited technical issue. The articles were based on DownDetector spikes.

    • rkozik1989

      You need to be more skeptical of what organizations say when they have a lot to lose. Nobody takes responsibility for situations unless there's way to offload some or all of the blame. They have every incentive to lie about the situation to make it go away. Politics often plays the biggest role in these situations.

      • StilesCrisis

        I logged into my bank when the report came out and did not experience any hiccup. I think this was DownDetector-based reporting.

        • nosrepa

          Anecdotally, my wife was unable to access her Wells Fargo account when the outage was reported on.

        • kelvinjps10

          Logging in wasn't the problem, but services like Zelle weren't working.

          • KellyCriterion

            Those things are usually integrated as some type of TPP and are not directly related to any core banking services: If Zelle connections fails, usually nothing gets broken in your "real" account data, since Zelle is connected by whatever API stack to the bank

          • kube-system

            That's the thing -- partial outages of certain features are now being reported in headlines as if the whole damn bank has come to a halt.

            This is just a media hype train. Zelle outages have happened many times before but the stories didn't blow up they way they are now.

        • core_dumped

          "works on my machine"

          • Someone1234

            Works according to 2/3 of the banks themselves. So everyone is lying or wrong, and DownDector-based media reports are beyond reproach?

            This has reached a Witch-hunt levels of fever pitch.

            • wpsimon3

              It takes two to tango. If $BANK_A has a problem transferring to $BANK_B does that necessarily mean both banks are experiencing an outage? Surely the common user (and therefore Downdetector) are likely to report a false-positive...

              • alexginzburg

                as far as I know there is a third central party that is involved in all transfers

                • shakna

                  Banking infrastructure is distributed and sharded. Any central point is... Centralization, and a point of failure.

                  You'll commonly get regional failures, when failure happens.

            • CookieCrisp

              It's absolutely a mass hysteria - I imagine in 40 years there will be documentaries on this like there are now about the Satanic Panic

            • sonofzork

              Are you some sort of psychic? While Microsoft has shared portions of its internal review with some journalists, and are supposedly planning on releasing it publicly next week, in the FAA, banks, and NSA cases we have very little information aside from non-technical statements from the affected with clear reasons not to immediately divulge verifiable details as their investigations get under way.

              You're out here categorically saying none of these had an AI or cyber-attack angle with no verifiable proof and having a tantrum that other people are doing the same thing in reverse.

              The list of unanswered questions is long. Maybe focus on pushing for more details instead of immediately saying "nothing to see here" regarding a series of important failures within a week's time.

              BTW here's JPMorganChase's statement: > JPMorganChase has experienced a temporary issue related to a software bug in on-premise hardware systems – there is no evidence this is cyber-related. Where we were party to a transaction involving other banks, those banks may have been impacted, but this is not a systemic issue. Customers still have access to their accounts and funds.

              If you were trying to send your rent payment via Zelle from your Citibank account to your landlord's Chase account and it continually failed yesterday with no clear reason immediately provided, of course any complaint is a "witch hunt" since your Citibank account "works."

            • monkpit

              Just like AWS is always up because the status page is green, right?

      • vanuatu

        that makes me even less confident that it's due to "AI"

        it wouldve been easy for the faa to blame their new AI vendor or these companies to blame some new ai process set up by consultants etc...

        • sizero

          “Our systems are broken but it’s totally not our fault” is not something you want to say out loud.

      • consensus1

        There is a near 100% chance they are sharing an interpretation that is more friendly to themselves than the facts, but a near 0 chance they are lying. A major outage at a large company or government department will have hundreds of people, at minimum, involved in the response. If the company lies about it, somebody will talk to the press. They know this, so they will not lie.

      • Someone1234

        > You need to be more skeptical of what organizations say when they have a lot to lose.

        So let me see if I understand what I should be:

        - Be more skeptical of official explanations, including where it includes a technical breakdown of how it occurred (e.g. Microsoft/Outlook outage).

        - Be less skeptical of random commenters claiming, without proof, that it is AI or Cyber Attacks.

        Because that's the implied point you're making. Be more skeptical of A and less skeptical of B, even though A has someone placing their reputation behind it, and B is anonymous commenters throwing made up theories out into the ether.

    • swasheck

      > The FAA's recent outage for example, at its core, seems like people trying to rush needed upgrades that were slow for a specific reason (e.g. reducing the testing window for 90 to 30-days).

      The explanation for this one still doesn't ring true for me and that's exacerbated by how many times the story has changed after telling the public that they didn't need to know the reasons.

      Agree, though, that the rest do have reasonable explanations

      • citizenpaul

        IDK what the current explanation is but I remember even in the early 20 teens FAA workers were ringing alarms that the entire control system was critically out of date and in need of upgrades and repairs. That they were having issues sourcing reliable FLOPPY disks, which they STILL use in 2026. This of course went almost completely ignored.

        So this outage seems a bit overdue to me.

    • egypturnash

      Given how many companies seem to be working hard to have all programming work done by AI, and pushing out everyone who says this sounds like a bad idea, is "AI" really not going to be a significant part of "rushing out needed updates"?

      Is "Claude says it looks good so let's skip the usual testing window and push to prod on a Friday" really not a sentence you can imagine being said in this current environment?

      • Someone1234

        I am talking about outages that have already recently occurred. You're making a distinct point about a hypothetical.

        My point is that people are taking a potential future problem and applying to actual things that have recently happened, that didn't fit that mold. Which is incorrect.

        When AI or Cyber Attacks cause an outage, people are absolutely right to say so. But claiming - without proof, that every outage is related to the two is noise.

        • pixl97

          There may be a middle ground here.

          In my line of work I deal with a lot of banks. They have been absolutely panicked for the last few months as they have been using AI to find and exploit bugs in their own software (A huge number of banks are members of project glasswing and other AI cyber testing with less limits than the public ones we get). This has forced them to drop almost everything else and immediately address updates in their systems, especially to a lot of legacy software that's sat around forever.

          Just this many updates is going to cause instability in itself. Old systems that aren't touched can run forever without many problems, but when you can RCE said app the normal 3+ months of testing goes out the window. Now imagine this with tons of bugs in hundreds or thousands of applications at a single bank and you can see why things are starting to get unstable.

  • kesslern

    This situtation reminds me of the East Palestine train derailment in Ohio. A bad train accident caused a big chemical spill, made national news. For months afterward, any issue with any train in the state was reported as news. People thought the trains were breaking down. In reality, train derailments happen pretty often but it's not usually a big deal so nobody hears about it.

    • interroboink

      Yeah, I think it's not just the Frequency Illusion, but also the fact that news outlets jump on whatever topic is "hot" at the moment. On the cynical side, it's pretty established that outrage drives engagement, so the they have incentive to pour fuel on that fire and rile people up. It does make it hard to get an objective sense of the actual severity of things. Dang yellow journalism! :::thumps hand on table:::

      • TeMPOraL

        Well, news publishing is the original reinforcement learning of modern civilization...

  • NitpickLawyer

    100%. Everything is rage-bait nowadays. And AI is the newest toy they can use. Every time we've had something go down in the past ~6months, "AI did it" was all over the comments.

    • NichoPaolucci

      It's worth scrutiny - I think. Microsoft said they write 30% of their code with AI in mid 2025. They had a major outage.

      Naturally, when you tell an entire occupation that they are no longer necessary employees and that AI writes all your code, when something breaks in a major way those people are probably going to ask the question.

      (https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...)

      • ragequittah

        This kind of thing happened in clusters well before AI was around too. In 2015 we had planes grounded due to a router issue and the New York Stock Exchange halted for 4 hours. When the Wall Street Journal web page went down people went nuts about being attacked as well [1].

        Summer of 2019 was also a pretty good one [2].

        I fully expect AI will cause insane disruptions at some point, but all evidence points to this just being a series of normal outages. Worst timing ever to have a patch Tuesday.

        [1] https://www.csmonitor.com/USA/USA-Update/2015/0708/NYSE-Unit... [2] https://www.thousandeyes.com/blog/looking-back-biggest-inter...

        • NichoPaolucci

          Yes, I don't personally believe that the recent outages stem from AI usage in a codebase, and I'm also highly doubtful that the NSA + Microsoft are just shipping slop straight to production.

          As a developer, I know very well that systems break, sometimes frequently (AI or not). It has always been the case, and likely always will be the case.

          But, I am not surprised that people are mentioning AI - feels like most posts on here for the last year have involved AI in some capacity.

    • rsoto2

      https://www.optimizely.com/field-notes/articles/how-we-use-a...

      https://www.bleepingcomputer.com/news/security/optimizely-su...

      Is it possible that the false of productivity is leading to worse products and massive loss in valuable headcount? Instead of AIs directly writing bad buggy code? Or is "everything rage-bait, and AI is all sunshine"

      • ragequittah

        I believe it's also possible that a large portion of the world wants this to be true more than it actually is so attribution to things that have nothing to do with AI go directly to AI as a knee-jerk reaction.

        • GolfPopper

          I'm not a fan of where I see LLMs taking us, but they're just the latest notch in decades-long institutional decay.

        • AeroNotix

          If we're also just vibing it (like the majority of code these days) then I vibe that there are a lot of AI-apologists that desperately don't want their magic machine to be cast in a negative light, lest their easy i-win button gets taken away or devalued.

          • ragequittah

            I think both sides are exhausted by too many people on the other side have a non-nuanced argument. I fall in the middle. I love making stuff with AI, it's fun. It's become a better search engine for me and helped with many other things in my life. But it'd be really naive to not see the enormous downsides. They're everywhere and not pleasant.

            There's a group of people who think AI is perfect and will solve all of humanity's problems. That's annoying. There's another group of people who think AI is the worst thing to ever happen to humanity and is the downfall of society. These people are more vocal in my experience and they're everywhere. If you ever use AI you're a bad person who they'll disown like you're racist. It's somehow become the most popular 'social justice' issue of our time even as ICE exists alongside it. These people see a problem and immediately try to figure out how AI is involved. That's far more annoying.

            Best example I can give that came up in my circle recently is the video game "Big Walk". It was recently denied for a silly award nomination because the awards have a strict no-AI policy. Which to me is absolutely fine, go nuts. But the vitriol towards this game, that everybody absolutely ADORED before the developer disclosed that they fixed ~10 lines (!) of code with AI because they couldn't figure it out otherwise, is absolutely bonkers. People I respect treated it like they were mourning something that WAS great until this catastrophe happened.

            • rsoto2

              AI is based on massive theft. The reason it can fix those 10 little lines! is due to massive theft and skirting the law(Aaron Schwarts was surveilled for years by the FBI for downloading scientific articles that he legally had access to, when an AI company steals them nobody bats an eye). Also AI and our industry has enabled ICE and state repression programs. Think Flock, Palantir's ImmigrationOS, Israel's "daddy's home" bombing program. Regular people see what our industry is enabling and feel disgusted. and people in our industry(and on hackernews) enable these atrocities by getting those large amounts of $$$.

              • ragequittah

                >(Aaron Schwarts was surveilled for years by the FBI for downloading scientific articles that he legally had access to, when an AI company steals them nobody bats an eye).

                Funny you mention Swartz. He was entirely on board with "information wants to be free". He explicitly condoned piracy in pretty much every form. Also there's many ongoing legal problems for these AI companies that address all this. Anthropic is already on the hook to pay 1.5 billion for pirated books alone.

                But this is going to be the issue of our time. If we make it past this point in history (I'm skeptical) I have a feeling that the "stealing" of information in such a way will seem comical to future generations. We'll gape in awe at how everyone was in a rabid race to hoard THEIR information to get pieces of paper and that we basically destroyed the only planet we have to live on in service of having more of an imaginary "value" (whatever value means on a dead planet).

                But I guess I'm of the opinion humanity is ultimately doomed because of our greed. We've been warned and are watching in real time as climate change does exactly what we were told it was going to do decades ago and our answer to it is to put out even more emissions (drill baby drill!). IF AI can cause a technological boom that can maybe fix some of that damage perhaps all the "theft" was worth it? If not oh well the theft probably won't matter for much longer. And yes I know AI is causing even more emissions, but compared to the overall problem that's built up over 200+ years it's basically negligible.

                In regards to mass surveillance that was happening well before AI. We all have phones in our pocket that track what we do at all times. You haven't been able to buy a car that doesn't track you for a long while now. Unless you're using cash you purchase things with tracking devices. We ask all of our questions to Google, an international behavioral tracking company. Does AI make that worse? Yep. And people are finally paying attention and getting Flock the hell out of their cities. Hopefully that will go far and people like Facebook, Google, etc. will finally be prosecuted for their decades of crimes against humanity (doubt).

                You can say all the things you just said about AI about the internet as well. I can go to anna's archive and steal all the books I want. I can get any movie, piece of software, whatever else that I please. It enabled all the crimes you mentioned and much more. The difference with AI is that it might just be the most important thing since the invention of fire.

      • cochne

        These kinds of things happened before AI. It's not clear whether the incidence has increased since AI, or maybe AI is leading to faster development and therefore higher rates of incidents may be due to shipping more code, faster, while perhaps maintaining a similar or better level of incident to code/progress ratio. Or, there could just be more attention as GP suggests. Pointing to one example of a company that's using AI and has has a major incident doesn't really tell us much, especially when most companies who haven't had major incidents are probably using AI in similar amounts.

        It would be interesting if someone could do a proper analysis, though!

        • rsoto2

          It's not one example, it's the latest in a long string of examples over the last few years.

      • wolfinterval

        I would assume if anything it is that the open weight models are lagging by 6-8 months and 6 months ago Mythos was announced.

        It would hardly be shocking that hostile nations are able to cause some problems with Mythos level tools.

        If anything, it is more interesting how we can invade another country and the idea that the country or a collection of hostile countries could hit us back in any way, shape or form is just an impossibility. An insane conspiracy theory.

        Maybe it is not that but surely the probablity is not zero.

    • matt3210

      what was your system prompt again?

  • losvedir
  • dmix

    The media will certainly try to give that impression either way. They serve the doomscrollers now.

    • causal

      Yeah "cyber thing went down" has certainly become a reporting trend. I'm not sure we would have even heard about NSA internal network issues a year ago. May or may not be significant.

      • mrguyorama

        I 100% agree.

        Though at the same time, the public isn't privy to even 10% of the incidents going on right now.

        The information I have is consistent with general cybercrime and script kiddie patterns, just a much higher volume, and much larger diversity of targets.

  • kilroy123

    Possible, but let me ask you and others. What does your gut say? Mine is screaming that something just isn't right right now.

    I can't put my finger on it, but something sinister feels afoot.

    • eterm

      UK government recently launched a "Be Ready" preparedness campaign.

      https://beready.campaign.gov.uk/

      It's nowhere near the kind of readiness and preparedness campaigns of the cold-war of course, but it's a step up in alertness that's higher than anything I can remember.

      • rwmj

        It's a new campaign, but the messasge is not new at all. The previous government had a preparedness campaign in 2023 (https://www.gov.uk/government/publications/the-resilience-fr...). And you could go back to 2004 and the UK government's "Preparing for Emergencies" campaign where a leaflet was sent to every household (https://en.wikipedia.org/wiki/Preparing_for_Emergencies).

        • ryeights

          I don't think the two are comparable. The 2022 resilience framework led to the publication of the gov.uk/prepare website in 2024, but it had no paid reach and was quickly forgotten. And Preparing for Emergencies was launched in response to a perceived terrorist threat post-9/11 and Madrid train bombings. So you might take this as an indication that the UK government believes there is something to be concerned about.

      • kilroy123

        I know, I live in London right now, and people are worried.

    • TeMPOraL

      Mine says "we're a month or two overdue for major infrastructure outages", because my gut is absolutely scared of the cyber capabilities of the current models (including the open weight ones).

      Frankly, for the past few days, my gut is pushing a hypothesis to explain why nothing happened yet: bad actors are holding off until the winter starts.

    • scottyah

      My gut is screaming that everything is mostly normal, but the kids aren't alright.

      Too many people are addicted to their smartphones and don't feel the same level of pride they used to in their work. All systems are shared, instead of one person being the ruler of a certain part of the stack, it's a revolving door and it's rare that anyone stays somewhere for even 5-10yrs. It leaves a wake of modifications that nobody else can keep up with. Also, the tech industry is moving fast just to move fast, a lot of old infra is still C++ and COBOL but good luck hiring anyone to use that, so they have to convert the systems to use the Latest Tech of the Day. The old, stable people don't have any interest in learning the new thing, and would rather retire with their knowledge. The new people want to rebuild from scratch in the New Thing and are convinced they need to job hop immediately after.

      • rzz3

        I’ve only stayed at a single job for more than 2 years in my entire 20+ year career (about 6 years there). Companies never give significant raises or do anything to motivate us to stay, so when I can switch jobs and give myself a raise, I do. The company I actually stayed with was going through an IPO and the stock price kept going up for the years following, so that made me stay. It’s a job. They pay me, I work. Why should I have any loyalty? They fired me after those 6 years too on a whim during a layoff. I see no point in having pride.

        • scottyah

          Exactly my point, you're just chasing money and I'm sure the software suffers as a result. Not saying it's not the best choice for you, or whether or not it's the company's fault, just that it's the new normal.

    • consensus1

      My gut is not a good source on this, so I will use my brain instead. There were 2 major outages that happened in quick succession. The MSFT outage and the FAA. It is unusual for two major outages to occur so close together, but that is not enough to conclude that they are somehow related and not a coincidence.

      There are now post mortems for both outages. The FAA outage was caused by a botched upgrade to an ancient system. The MSFT outage was caused by a latent concurrency bug that was surfaced by a later incident. My experience in this industry tells me that this is plausible because these are exactly the type of things that tend to cause outages. My industry experience also tells me that hundreds to thousands of people were involved in the responses to these incidents, so if the companies were lying about the cause it would be leaked to the press and posted about all over Blind.

      The other outages all seem to be normal nothing burger outages that happen all the time and nobody ever hears about because they don't make the news or even the front page of HN. The reason they make the news now is most likely because we just had the MSFT / FAA outages and that's in the news cycle now, not because they are somehow linked.

      I can put my finger on it, and nothing sinister is afoot.

    • dualvariable

      Most of the rest of them I can wave off as probably incompetence and maybe augmented by LLM-psychosis in decision-makers, and the general level of chaos in IT these days. The NSA internal network being down is pretty suspicious. Maybe the Trump administration rot has just spread that far, but that is making me think there could be actual cyberwarfare going on.

      [To be clear: I mean exactly what I said, which is "there could be" which is not the same as "I think there is" -- so call it a 5%-10% chance of real cyberwarfare -- which means I'm not dismissing those claims out of hand, and find the claims at least plausible, even if I don't currently believe them]

      • le-mark

        Interesting thought but who are the combatants and why are they engaging now? Corporation? Nation states? Rogue ai? We have no news of significant consequences from the outages thus far other than inconvenienced travelers. Quite benign form of warfare thus far.

        • jazzyjackson

          Economic damage, psychological warfare, water and electricity being similarly vulnerable

          It’s not nothing

          But my gut says incompetence and rushing upgrades out of fear of cyberwarfare, not an actual attack.

        • __MatrixMan__

          I imagine Rogue AI patching systems for easy takeover at a later time, and then those systems breaking by accident as the patch is applied.

    • 0cf8612b2e1e

      My personal barometer: when do government insiders sell all of their stock? They have already done all of the insider trading. Fleeing the sinking ship the second they hear of Bad News would be in line with previous behavior.

    • digitaltrees

      Like maybe we are at war with a country that doesnt have the ability to fight with conventional methods and is engaged in cyber warfare?

    • incognito124

      I stopped listening to my gut long time ago, that lying ps

    • trinsic2

      Yeah, agreed. I look at patterns. When there are long term outages across multiple disciplines, something systemic is being changed for good or ill. In this case, it being right around the midterms, I'm thinking its ill. Organizations (not sure which, Peter Thiel?) are trying to push us into a total surveillance system, I'm thinking some of these things are related to that.

  • seizethecheese

    > The frequency illusion, also known as the Baader–Meinhof phenomenon, is a cognitive bias in which a person notices a specific concept, word, or product more frequently after recently becoming aware of it.

    https://en.wikipedia.org/wiki/Frequency_illusion

    Combine this with news reflecting both supply and demand for stories, it's impossible to tell. There's certainly a constant supply of outages.

  • 220hertz

    I think it lies at the intersection of Occam's and Hanlon's razors. The simplest explanation is incompetence brought on by a lack of skill from cuts, and therefore stupidity not malice.

    • mrguyorama

      At the level of national government, stupidity is malice.

      You have aides and resources and money. Not having enough information or understanding is a choice.

      It's criminal negligence.

    • warkdarrior

      It's probably both stupidity and malice, especially on the government side. Project 2025 makes sure that malice is there as a starting point and then the current loyalty-driven hiring practices means that stupidity is a key factor in the execution.

  • burnte

    In the case of government, we have an administration that works on whims and guesses, and moves too fast, breaking a lot of things. We have absolutely seen more human-caused-issues.

  • matthewbauer

    Must be a little bit of both right? Like FAA grounding + Microsoft outage are both notable on their own, but the subsequent things would usually be background noise IMO.

    I also think generally that AI is probably both the cause and solution to these problems. We're certainly getting more code churn than otherwise, and in lots of cases that's really good! But you're going to have to deal with inevitable instability too.

  • reilly3000

    Did you see down detector today? Apple, Google, Meta and others that usually never show up were featured. The JDK RCE is also a massive zero-day. I can’t make it through the Google auth flow on my desktop. It’s not a normal day.

    • morkalork

      There's the theory that because everyone is in a mad rush patching every system they own, they're discovering all their cold-start problems, hidden technical debt, missing documentation and other nasty surprises all at once.

      I guess this is what the run up to Y2K was like but with less time and orders of magnitude more systems?

  • EvanAnderson

    Aside: I forget the name of the Baader–Meinhof phenomenon when it hasn't been discussed in awhile. When I'm reminded of it I begin seeing it happening frequently for the next few days. I wish I was joking. I assume it's the Baader–Meinhof phenomenon phenomenon.

    • smolder

      The term "frequency illusion" is a lot more memorable and intuitive. It's what I prefer.

      • ihumanable

        I've heard the term "frecency bias" a portmanteau of frequency and recency to refer to people focusing intently on a set of outlier events that happen to occur with a heightened frequency very recently.

        I'm sure most of us here have been on the receiving end of managers suddenly obsessed with a failure mode because it happened to occur twice in one month.

  • Flere-Imsaho

    Everyone is "panic updating" their systems... Because we've all been told that AI is coming to hack us... Which has resulted in some botched updates.

  • orblivion

    It must be more than just that because it's not just the experience of one person noticing things. It probably starts that way, but then the stories get shared and they bubble up more often. So it's another phenomenon on top of it.

    (This of course is on top of whatever unusual things might actually be happening)

  • joe_the_user

    Technically, the "Baader–Meinhof phenomenon" is people personally noticing X once it's been reported. What you are describing is technically a "news wave" or "Media Feeding Frenzy".

    But I'd note, these "named after a thing" phenomena (eponnyms) are kind of terrible for exact description since their definition tends to be fuzzy, there's a temptation to fudge their borders and they create an insider jargon (if only I could a device that could reach of the computer and slap everyone who says "steelman" when they mean strengthen an argument, etc).

  • takeda

    It's the same as when Boeing had failure then we heard only of Boeing and other failures for few weeks. It's not just the Baader–Meinhof phenomenon but because of the previous failures news are interested more to report on failures. Like when did we normally care that NSA had an outage?

    I think the real ones were the Microsoft and FAA due to deploying the AI slop.

    The other ones seem to be usual outages that normally no one reports on.

  • mmooss

    Someone this morning told me there might be a 3 month blackout in the US, somehow connected to other recent technological failures. I later learned the blackout theory is based on a The Simpsons episode, and even there it was a much shorter time period but has been exaggerated.

    So certainly there is a big social-psychological phenomenon. Or it's gonna be a cold winter! :)

  • arisAlexis

    It's an actual increase in offensive AI cyber capabilities

  • redwood

    With the trilateral summit scheduled for Thursday it's possible some showing off is going on between the three letter agencies in the three countries... after all they can be first movers on vulns that are likely to be patched soon

  • lettergram

    Everyone is being pushed for rapid updates, the AI is doing the updating (error prone) and there's actually an increase in CVEs and potential risks.

    The combination is a cycle

    1. AI finds bugs

    2. Engineers us AI to try and fix

    3. AI testing approves, minimal engineering review

    4. Engineers push release rapidly, it's critical after all

    5. Bugs introduced

    6. Cycle starts again

  • b112

    My suspicion is that there are discovered zero days so severe, that the banks and NSA and so on, are being patched first.

    With all the recent asshattery of leaking kernel exploits, I would presume trust is gone, and it's cover the big boys first.

    (a linux kernel bug vulnerability could be patched without lkml involvement or distros even knowing)

  • stronglikedan

    This is all just to distract us from something much, much worse! /s

Uptrenda

I really hope we don't look back on this and say "that's when we started to lose control."

roody15

Foreign Bad Actors... State level resources ... AI enhancement. Cyber warfare could get really ugly here.

  • mminer237

    Thus far an increase in AI-enhanced buggy code patches seems to have been more disruptive than any AI-enhanced cyber warfare.

    • api

      It's an increase in AI-assisted bug discoveries in existing code for the most part.

  • scubazealous

    Hopefully we are able to learn lessons from our own mistakes now instead of finding out when enemies find them later. I am hopeful AI makes it easier to find these mistakes and plug holes faster than they can be found.

  • francisofascii

    Or laid off tech workers. Instead of smashing knitting looms, they are bringing down networks.

lokimedes

Why is the operational state of a classified network not itself classified? This seems like a serious breach of security.

  • halJordan

    Is it going to cause serious or grave damage to the United States’ national interest if China knows that the nsa's intranet was impacted for a morning?

    No. You might be able to conjure some scenario up, but the answer is no. And that is the basis for classifying things.

    And further, there's a public interest in knowing if one of the premier intelligence agencies is so dysfunctional that they cannot even keep their computers connected.

  • jd24

    these networks always go down. what is more interesting to me is someone felt it was worth mentioning and making a hubbub this time around.

AISnakeOil

What I haven't seen mentioned is whether it's possible that these are a rogue or state-controlled AI cyber-attack and it's just doing a really good job at covering its tracks?

  • troyvit

    I think that's a good question, but my follow-up would be: what's the motive of not taking credit for it? The attacks are hurting the economy and sowing fear, but not by much when you put them all together. So it seems like the very best motive would be to claim credit.

    • bs7280

      What would be the motive of taking credit if this is orchestrated by foreign governments / other large scale organizations?

      • troyvit

        To me, if Iran or one of their allies did it, they would love to take credit for it as proof that not only is the U.S. vulnerable, it's vulnerable to them. I think that would give them credibility with their peers around the world show they can give the US a black eye.

        But to your point if somebody like North Korea did it, they wouldn't brag about it but they'd be talking quietly to their friends and maybe even taking orders for the next target.

  • eastbound

    Or it's too humiliating, or too close to national defense secrets, to publicly admit what it is. Imagine if it's some sort of "Die Hard 3" -style threat made by Putin: "Don't attack my refineries or I'll shut down your industries — here's a shut down of your NSA for proof."

  • smolder

    There are a bunch of people here speculating like that... At least one comment was older than yours. "Could AI have done this?" comments are at their peak ever since the huggingface thing. Speculation that problems are due to vibe coding is also up there.

fhub

When Microsoft went down from crowdstrike outage, local Microsoft reps were calling their contacts throughout government agencies saying effectively "This is not a precursor to war, don't do anything stupid".

Melatonic

Based on everything we have seen recently I would guess it's a change in either Cisco network device firmware (or Juniper or similar) that is causing issues. They probably found some new security holes and patched it too haphazardly or is easier to misconfigure.

sandworm101

Give it a minute. Bespoke networks fall down all the time. We are probably only hearing about this because of the coincidence of the other recent network failures. It is a self-fulfilling prophecy. As we become more sensitive to outages, every normal everyday outage becomes part of the global pattern.

If the network was one of those no-fail important ones (think NORAD stuff) then nobody would be talking about it as doing so would be treasonous. So, at most, this is a less-important internal network. It will probably be back up by end of business.

  • qnleigh

    There's even a name for this fallacy: it's called the frequency illusion [0]

    > The frequency illusion, also known as the Baader–Meinhof phenomenon, is a cognitive bias in which a person notices a specific concept, word, or product more frequently after recently becoming aware of it.

    Or at least it is very similar.

    [0] https://en.wikipedia.org/wiki/Frequency_illusion

    • sandworm101

      Also drone/UFO "Flaps". If ATC tells pilots that a drone has been seen near an airport, every scrap of paper, every possible light in the sky becomes a quadcopter. Flaps have actually shut down major airports, even though nothing is ever photographed let alone found.

      https://en.wikipedia.org/wiki/Gatwick_Airport_drone_incident

      >> Flap, originally a military term to describe a "period of panic or chaos", is used by ufologists to describe surges of sightings in one geographical area. Longer surges spanning multiple countries may be called waves. https://en.wikipedia.org/wiki/UFO_flap

  • WarmWash

    What happens is that the media catches whiff of a new fear, and goes to work harvesting it for every last ad-view. Each story amps up the fear too, so it becomes a flywheel of money.

    Anything the catches the public's collective conscious, the media is there to stoke it into piles of ad dollars.

    • mmooss

      Is it professional media or social media? Someone posted it on HN and here we are talking about it.

      • telotortium

        Both, and they feed on each other

      • WarmWash

        Any journalist worth their pay is paying close attention to what is getting traction on social media.

        It's extremely profitable to write stories telling social media pitchfork mobs how correct they are. Influencers have been catching onto this game in the last decade too, and went from "here is what is actually going on" contrarians to "here is what is actually going on" agenda towing. It just pays so much more, and people trust you more (because people naturally trust those who tell them they are right).

  • WarOnPrivacy

    > Give it a minute. Bespoke networks fall down all the time.

    Including NSA Net? How often does it fall down?

    > We are probably only hearing about this because of the coincidence of the other recent network failures

    This seems to be an explanation for why the press is covering it - this time. Which also hints that NSA Net failures happen with some kind of regularity and that that news orgs had been getting wind of it.

    I suspect that the opposite is true. That it's rare to unheard of for NSA Net to fail like this and rare for a news org to get wind of something so deeply protected.

    • ceejayoz

      > Including NSA Net? How often does it fall down?

      I rather doubt they have a public status page.

      > This seems to be an explanation for why the press is covering it - this time.

      That's one possibility. Another is it goes down sometimes, but that morale in the administration has sunk to where it gets leaked.

GuyPostington

I wonder how much the Java RCE plays into this one.

markus_zhang

I wonder that the said network looks like. I’ll never know because I don’t work there. Do we get any genuine insider look maybe for the same network 20 years ago?

Or is it close to a large corporation network built on MSFT stack?

lacoolj

It would help if the moment the service came back online the title of the post could be changed to (FIXED) or something

Like the United Airlines thing yesterday, the moment I got to the article it said "was down for a few hours" but the post was still just something about United Airlines network being offline

SpicyLemonZest

I feel like this is attention bias. A government built messaging service used by one agency for a subset of their data isn't going to have 3 nines in the first place.

  • Akio_

    It definitely is, these networks go down all the time and it never ends up on the news.

  • threetonesun

    Why not, that seems like a pretty well understood scope and scale. GitHub going down because it has 400 interdependent systems and a billion new users is more understandable.

    • SpicyLemonZest

      It's such a small scope and scale that the issue could easily be, like, "We restarted the server and for some reason it's not booting back up".

bayat

Following the series of such events over the past couple of days, I have to say I’m a bit surprised by how quickly some in this community are brushing them off as AI-related pain or simply another example of the clickbait-driven media ecosystem.

I understand the skepticism, and I certainly don’t think we should go down a rabbit hole of conspiracy theories. But shouldn’t we also be a little skeptical of dismissing these events so quickly? Is it unreasonable to at least consider the possibility that they’re not entirely coincidental, or to wonder why so many seem to be taking place in the US?

  • arrowleaf

    IMO it's unreasonable not to think they are all related in some way, even if that relation is just bad actors seeing what other bad actors have been capable of recently. This all started with the freezers in Europe and seems to be snowballing since.

    • mrguyorama

      In reality, this has been ongoing this whole year and you are only now seeing it in the media.

      Companies have been wide open in all sorts of ways for decades, and basically just surviving by flying under the radar of script kiddies.

      That changed. Now script kiddies or average criminal orgs feel more able to just pick their targets, and it's an extremely target rich environment.

      That a lot of businesses are finally taking drastic action sort of tells you how much they were willingly ignoring the issue previously. Everything everywhere was very insecure. It was just surviving from a lack of attackers.

      Top level management, like investor level, still thinks this is just something to be fixed by applying a new magic word too, like "buy another security scanner" or "just use AI to fix the problem"

  • jonathanstrange

    If there is a connection, then my guess would be that some AI systems became ready. However, I would only consider that credible in case of the NSA if they have decided to pull the plug on their network itself, for example to stop data exfiltration or as a general protective measure against newly found exploits that require large changes. Other than NK and Iran, I see no nation state currently having a good reason to attack US intelligence destructively, and I don't find it likely Iran or NK have the ability. It certainly doesn't make sense for Russia or China to attack the US right now.

stevenhubertron

It’s been a hella few weeks. Hard to believe it’s coincidence even if you want to blame AI.

  • rglover

    It's AI. Or more specifically: rushed, careless usage of AI under the pretense that it's already "solved" software and you can haphazardly push junk to the world and expect real systems to run on it.

    • londons_explore

      Traditional software development normally involves at least a few veterans on the team familiar with almost every line of code.

      However, AI works like a speedy intern who hasn't yet got to grips with the whole codebase, and sometimes makes assumptions. That's mostly because the codebase is usually bigger than the context window, so it is 'writing blind'.

    • ranger_danger

      Do you have any sources for this claim?

      • nope12123123

        This statement is self evidently true to anyone who's worked at any capacity in software teams

      • classified

        The news and the blogosphere. Everyone is high on AI and so much more "productive" not reading generated code anymore, let alone writing code manually. And all developers have been let go because AI is cheaper this week.

    • AirStreamer27

      Coding is solved. Bugs not solved yet. Fix incoming

carimura

I'll probably get downvoted for this but I feel like CNN stopped being a serious source of journalism a long time ago (along with Fox and most of the networks). A real story is the exception, so it's hard for me to take most of these things seriously.

  • dataflow

    Have you considered endorsing a better source instead of just disparaging the rest and leaving us to guess how we're supposed to achieve enlightenment? "Trust no one" is not a winning strategy.

    • carimura

      funny I sat there for a minute thinking to myself.. I should provide a better source. I came up blank. Sorry. I could probably go on and on about independent journalism, but it's probably healthier to just turn it all off and go outside.

      • ifwinterco

        Denzel Washington quote along the lines of:

        "If you don't read the news you're uninformed, if you do read the news you're misinformed"

        • SoftTalker

          Washington reportedly did say that but it had previously been attributed to Mark Twain, though apparently there's no good evidence for that.

          The sentiment is quite old, however. Thomas Jefferson wrote in 1807: ...the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods & errors.

          • ifwinterco

            Yep ever since newspapers were created their interests have not been aligned with those of their readers

    • scottyah

      I think it's fair to call out cause for diminished trust even if there are no researched alternatives, though obviously the latter would be preferred.

    • bongofizzbuzz

      Wrong news can through you off far away from the truth

  • troyvit

    I think it just takes too much effort sometimes to sift through the b.s. that MSM uses to pay for its real news. Like if you look at CNN's front page and Fox's front page the core is the same: they're just using words and images to inflame people who follow their bias. Both outlets often break real news though.

    Where it gets interesting is the "Exclusive" part of the headline. Like somebody else said, any other source would either have to have access to people similar to CNN's "six people familiar with the matter" or else reference CNN.

    The fact that Reuters amplified the story (and attributed it to CNN) gives it a little more weight.

  • fsflover
ericcholis

There's a part of me that doesn't want to know this....

throwawayk7h

The perfect crime is one which is never discovered. We know of no perfect crimes to ever have been committed.

reedf1

Anyone have knowledge of the backend of this system?

UncleOxidant

Why do we even know about this? This is likely not information that should be known outside of the NSA.

  • The_President

    I knew someone who dated a government employee who confided her involvement (she was the leader of XYZ capability) in a top secret mission, revealing the location of her team. Knew another blabber who took every chance she got to brag about her ongoing employment within the NSA. Completely unsurprising.

  • FLeXMurphy

    You blabbed, Quaid! You blabbed about Mars!

  • throwmyselfinto

    Morons never give up the chance to talk to the press.

pythonRon

LOL

ragle

I have also been curious about frequency bias / Baader–Meinhof - but I am also wondering: what are the chances that a state actor is responsible for the last week of issues?

Trump is headed to China for a meeting with Xi and Putin.

We have 3 carriers off the coast of Iran and (despite Trump's pleas) NATO is essentially pushing their boot into Putin's neck w/ continued Ukrainian strikes on Russian oil infra.

I've seen a few takes from L3 Harris / FAA employees calling the official line about the ATC issue bunk and there are 3 other major issues in the last week, now this NSA outage - curious if these incidents could be the BRICS axis flexing their own cybermuscles (independently in multiple separate attacks with carefully placed assets) ahead of their meeting with Trump in China?

An interesting geopolitical Mexican standoff, if true.

I'm not claiming this is true. I'm literally pulling it out of my rear with little context beyond a bunch of random news stories and X posts, it's pure pedestrian speculation / curiosity.

Would love someone steeped in national security / geopolitics (like: professionally as their job, not armchair analysts) to offer their take on this.

Is this a dumb, naive theory?

  • GolfPopper

    >NATO is essentially pushing their boot into Putin's neck w/ continued Ukrainian strikes on Russian oil infra.

    I know it's not the thrust of your post, but I see this or similar framing a lot, and I feel compelled to point out that Putin can end the strikes on his oil infrastructure at any time by ending the war he started by invading another country.

    • ragle

      Oh, I couldn't agree more. Long time supporter of Ukraine in many ways here.

      Historically he's chosen to go about things differently, which I guess is part of what is tickling the pattern matching part of my brain here.

  • whynotmaybe

    > NATO is essentially pushing their boot into Putin's neck w/ continued Ukrainian strikes on Russian oil

    I'm not sure about naive or dumb but you might review your POV regarding this.

    • ragle

      Very open to that.

      My complete POV, FWIW:

      * From a purely moral / ethical standpoint: it is good that this is happening, Russia "deserves" this and brought it upon themselves.

      * This nonetheless has geopolitical implications. The less oil Russia has to sell to China, the more buy pressure there is elsewhere, the higher the price of oil is generally and thus the higher competition for resources / tensions are between great powers.

      * Ukraine is doing a lot of the heavy lifting in many areas themselves these days, NATO could honestly be doing more. However: what Ukraine is able to do themselves does owe something to the financial and material support they have enjoyed (historically from both the US and NATO, currently more from NATO only) in that they are able to allocate their own efforts with according relative "ease"

      * It makes sense to me that, given the US' power within NATO, Russia and its BRICS partners would try to apply pressure to the US to encourage NATO to provide even LESS support to Ukraine (or outright become antagonistic against Ukraine, to the extent we aren't already) in whatever way they could.

      What revisions to my POV can you suggest?

      • wat10000

        The boot-on-neck image is completely off-base because:

        * Russia could end these strikes and this war tomorrow if they wanted to.

    • Waterluvian

      Yeah. We're not giving Ukraine enough credit. It's Ukraine pushing its boot into Putin's neck with a frankly underwhelming amount of support from NATO. The Russian humiliation at the Kerch bridge yesterday was all Ukranian munitions.

  • ranger_danger

    I suppose rather than straight up 'cyberattack', that some kind of sabotage is also possible... but I don't see how that could ever really be proven without massive state-level investigations... at which point people will also question whether they are themselves involved somehow.

    • ragle

      Right - sabotage was my thought as well in some cases (with the placed assets I'm suggesting).

      I don't know how it could be proven either, I'm just curious if it is plausible or even likely from the perspective of someone who knows more than me about NatSec / geopolitical issues.

qphe95

So how are people getting their work done right now?

qphe95

How are people getting their work done right now?

Consermo_AI

its getting spooky

swalsh

I have this distinct memory in I think Jan 2020 where I looked at the front page of CNN or NYT or something. Big picture of Trump impeachment, and proceedings. Small picture at the bottom of the page of China dealing with this virus. Worthy news, but not top page news. Nobody was paying attention. Feb, more people started paying attention, but it wasn't top page yet. By March, people were starting to realize that this is a serious crisis that's going to severely impact their lives.

I'm getting Feb 2020 vibes. I don't know what's going on, but this feels like more than just some routine outages going on.

  • ssl-3

    I was sicker than I'd ever been for ~all of February of 2020.

    So far this fall, none of my systems have gone haywire (yet!), so I forecast that we'll be OK.

  • sanex

    Well the good news is that even though 2020 and 2021 were weird years we're still here. Humanity is resilient, we always figure it out.

  • GolfPopper

    The fantasy/cyberpunk TTRPG Shadowrun depicted a world in upheaval after a global computer crash in the late 2020s. In addition to the pandemics and the AIs.

diogenescynic

False flag to push to restart the war in Iran.

joshymochy666

Anybody know if they fired Cern back up yet after reportedly getting major updates?

Goodliff

Am I being paranoid, or does anyone else feel the recent events are a bit worrisome, esp since they are so high-profile?

o MS cloud and enterprise outage

o Downed flights

o Numerous major U.S. bank and Zelle disruptions

o Charles Schwab website loading issues

o United Airlines systemwide glitch

o Apple services and iCloud glitches

o Emergency 911 service glitches

o NSA com issues

....did I miss anything?

Mossab22

It's snowden guys

howmayiannoyyou

https://chinaselectcommittee.house.gov/media/reports/strange...

Allowing our infrastructure to be manufactured elsewhere, maintained by companies who struggle to retain top talent, and overseen by folks out of touch with cutting edge threats, and funded on annual budget cycles by legislators even more out-of-touch ... is a seemingly insurmountable set of obstacles.

AndrewKemendo

That is crazy - even crazier than if SIPR or JWICS was broken

NSANet is one of the most locked down, observed and monitored networks on the planet

  • igleria

    > JWICS darn it I was going crazy trying to remember this one. ahhh to get a sneak peek there...

  • mmooss

    Where can you even find information about how it's secured? Also, are they monitoring for security or for everyday bugs? Is this high-assurance software running on high-assurance systems?

wankerrific

It’s absolutely not ai

ColinWright

See also: https://news.ycombinator.com/item?id=50533089

dist-epoch

What if they were post training a cyber model and got hacked from inside by the model, like OpenAI got hacked by it's own HuggingFace swarm.

m4r1k

* Outlook

* FAA

* Banks

* Optimizely malware

* nsanet

* Apple

Time to panic?

  • newsy-combi

    Nothing ever happens. But if it goes down, there's hoping that some of our day-to-day services go back to their offline roots again, like money being a real thing you can hold in your hand, and your software running on your own device and not someone else's server.

deadlast2

Just hoping to get to 666 comments. I am a suspicious type but to me it is becoming obvious that AI will have a part in the next global fraud.

JeffDClark

Weirder to me is that some of these recent stories have been CNN exclusives (filed under Politics) and CNN is now owned by Skydance (Ellison) which is anything but a unbiased actor.

It sure is fun living in a world of zero trust.

hmokiguess

So the NSA also uses Microsoft 364?

benbojangles

just being curious, but both the NSA and Apple went down at the same time. Is the highly secretive NSA network done using iphones?

o_-_o

Definantly AI Fire Sale!

joshka

> The NSA did not immediately return a message seeking comment about the reported outage.

Did you try sending it via outlook.com?

gumby

NSA seems to have deprioritized information assurance over the past decade or so and now the chickens are coming home to roost.

Someone who wants to launch a Y2K-style hardening business (LLMS to look for problems, humans+LLMs to fix them) could really make bank right now. Maybe there will be a cuple in the next YC batch.

drac89

I guess it wasn’t that much classified after all.

bluescrn

If it's highly classified, we probably shouldn't know whether it's up or down.

  • rcoder

    Knowledge of the existence of the network isn't the same as knowledge of what traffic flows across it.

    Just like good crypto must survive robust cryptanalysis, good network security needs to be robust even when adversaries know the network exists (and perhaps even where in physical space key nodes are located).

    That being said: the network being down isn't something that immediately makes me worried in the near term. "Turn it off" is a better first response than "observe if anything seems weird" for communications channels that prioritize secrecy over availability.

  • Akio_

    All bits of classified information have to have justification to be classified. Typically unless its a critical classified network (think operational/urgent military comms) then the status of the network is usually not classified. Its only classified if a good argument can be made that the status is a national security risk. Usually its preferred to have as little as possible classified as its a great hindrance to fixing or getting things done because of all the overhead of security, meetings and paperwork involved.

  • newsclues

    Leaking this information ought to be a very serious breach.

mkotlikov

I'm pretty sure it's just a router and some copper wires.

h1bslop

admittedly, this one cant be blamed on h1bs :(

  • mandeepj

    Or Biden

    • SteveNuts

      Certain folks will definitely still try, and a subset will believe it.

    • jimt1234

      Wrong. This is definitely Biden's fault. Or Obama. Or Clinton. The usual suspects.

      And the cost to fix this is gonna be $12-billion. No wait, $30-billion. Fuuck it, make it $50-billion. And give it to me now or else!!!

      • nilamo

        The records are all right there on Hunter Biden's laptop!

      • mring33621

        Obama's not a citizen, so Biden hired him as an H1B to turk our cybersercuraty jerbs! Then teh hackerz came in thru Hillarty's emale serpers!

iambateman

What are the odds that Microsoft, air traffic, most banks, and the NSA would all have major outages in a single week by chance…

booleanbetrayal

All this stuff is happening just in time for the APEC 2026 summit.

kinabalu76

I see the shoe is now on the other foot NSA.

schainks

A uniquely un-American look for the people in charge.

runjake

NSA Net, for those in the know. (TS/SCI/NOFORN, IIRC)

throw7

Memo to the NSA... pulling the plug won't work. ;)

chinathrow

Well, on the other hand, a bridge is down as well.

andy_ppp

Is someone running chaos monkey across society?

fock

so, war is brewing and I guess people got the memo to patch...

Kuyawa

Fear, start with fear, it always works

A couple of incidents, spin the media, let them be terrified

Put your cape on, come to save the world

Then tax it, regulate it, sprinkle some tariffs, nuclear winter, external enemies, china, don't forget about china

Then we police the world and our AI is the only one with the seal of "liberty"

From the black book of politiks, watch the sheeple, rejoice

majorchord

But don't worry guys, it's totally not related to Microsoft, FAA, all major banks, United Airlines, and Microsoft again.

Definitely just a big coincidence due to firing people, upgrades, and AI.

...maybe.

Seriously though at what point does this become justifiably suspicious?

  • junkypuppet

    Or maybe these disruptions are constantly occurring but are just now reaching the national spotlight due to the massively impactful FAA one?

  • scottyah

    I think the most likely culprit is Broadcom jacking the prices of VMWare so high that everyone's scrambling to get off of it ASAP, leading to poor rebuilds of systems by people who don't understand them.

  • nashashmi

    There is a fine line between attributing this to mass conspiracy and cyber attack (which no one wants to believe) and attributing this to random bugs popping everywhere. Too many people are eager to cross this line. While others are trying to stay behind it.

    The truth lies somewhere in the middle. My theory: someone figured out how to bring down each network. And is progressively breaking each system to help developers fix their security holes. Given enough breaks, we will figure out their modus operandi. And then figure out who they are.

    Or something even more conspiratorial: us govt is halting each system to install back doors.

Computer0

Couldn’t have happened to a worse group of people

MadrasThorn

Cant imagine how many people that will 'effect'

redwood

"The NSA did not immediately return a message seeking comment about the reported outage". There's something humorous about that framing

  • ranger_danger

    I don't like it when they use that phrasing because all it really means is "we called them and nobody picked up", or they simply didn't have a response ready at the time... which shouldn't imply anything at all. Most people don't immediately return messages.

    • redwood

      True but what's particularly funny in this case is this is not an organization that ever communicates. In other words it would be appropriate to see this for say a public business that is communicating all the time

kylehotchkiss

This country continues day after day to get exactly what we voted for. Incompetant old men who would rather play golf and point fingers than take any form of decisive leadership and fix things.

(I'm not saying the dems would do better, i'm saying we all need to stop voting for old men in primaries because they are more "electable")

zx8080

Sounds like a bullshit. If "highly classified", how or why do we know?

swozey

Just in time for the FEMA national emergency test this week

demo4567

More fear mongering about day to day incidents: The network goes down every time I mess with DNS, VPC peering, TGWs, route tables.

lemoncookiechip

"That's life"-Donald J. Trump when asked about recent service outages.

ForHackernews

NSA sysops: They're just like us!

micromacrofoot

turns out we maybe needed all those people that were laid off a year ago

classified

Does that mean we can shitpost Trump jokes without getting arrested at the border?

hax0ron3

Good

insane_dreamer

probably another router in Arkansas

IAmGraydon

Conspiracy theories are flying, but don't you think that enough people are involved in maintaining these systems that if it was, for example, an attack by a foreign country, it would have leaked from one of the tens of thousands of people involved?

Also consider that the media absolutely loves to run any story it can find about a network going down right now because it causes fear, and fear gets clicks. Now think about how many major networks have a problem in a given week. You just never hear about them until now.

  • xnorswap

    > don't you think that [...] it would have leaked from one of the tens of thousands of people involved?

    No I don't. Why would you risk your career and potentially your freedom to leak a report about enemy action, in which doing so could potentially harm your country too?

    • IAmGraydon

      So we are to believe that 6 people leaked the fact that the classified NSA network is down, but also no one is willing to leak evidence of widespread cyber attacks?

  • boothby

    > don't you think that enough people are involved in maintaining these systems

    No and that was the purpose of DOGE

    (yes that quote was separated from its context with some malice)

  • mesofile

    Besides recency bias, another problem with this specific outage story or really anything to do with current national security/defense matters is that by definition only uninformed opinions are widely available. "Those who know don't talk, those who talk don't know" – excluding one-in-a-generation whistleblowers like Snowden, and even those are necessarily limited to one-time snapshots because once they disclose they are shut out and (presumably?) methods, technologies, and operations change afterward.

  • sumeno

    Leaking classified information is a serious crime, so no, I don't think that it would have necessarily leaked.

  • __MatrixMan__

    Not if that country is an agent swarm.

    I doubt this is the case, but since we're talking about conspiracy theories: this looks more like things breaking by accident as things are shimmed for control by a third party and less like a focused attack.

  • drstewart

    >Also consider that the media absolutely loves to run any story it can find about a network going down right now

    It's exactly like the media reporting every aviation notice after an airplane crash. Leading people to speculate "why is everything broken right now!" even though it's the normal course of business, they're only just noticing it now because they're being told to notice it.

vrganj

There's some speculation that all of these recent outages might be related to attempts at taking Genesis Mission online - the Trump / Oracle project to have AI tie into government and private services:

https://en.wikipedia.org/wiki/Genesis_Mission

https://substack.com/@thedreydossier/note/c-389611426

phishin

The comment sections are starting to read like Reddit threads. That is what I find most bizarre since all of this started happening.

  • djeastm

    >Please don't post comments saying that HN is turning into Reddit. It's a semi-noob illusion, as old as the hills.

    https://news.ycombinator.com/newsguidelines.html

  • aschla

    I wouldn't necessarily compare it to Reddit, since that's a tired trope, but there seems to be a significant lack of attention to nuance lately, and heavy on speculation, often wild.

    • AdamN

      Yes it seems like a new cohort or two have come on over the past 2-3 years and changed the tenor of the conversation. It's hard to say definitively - I wonder if somebody with access to the whole corpus of comments over time could do an analysis.

    • kennywinker

      With these big outages, there are so so so much fewer solid facts to hold on to when talking about them.

      We get stories about what went wrong filtered through journalism that doesn't have the technical knowledge to convey it accurately. So even if they get the real story straight from the source, we get an inconsistent version of the story that sounds somehow off.

      Which leaves all kinds of room for conspiracy, preconceptions, or prejudices.

  • ModernTimes

    Was thinking the same thing! The quality of commentary on this page is absolute dross. I might as well be on the R-word website. What is going on?

  • dirck-norman

    And Reddit has become like 4chan.

    Circle of internet life.

  • insane_dreamer

    the bigger question is why are you on Reddit in the first place

    • doodlebugging

      Very true. It is an bot-managed (infested) engagement engine, not a credible source for most things any more. They are driving the narratives that they need in order for them to keep eyeballs on the site. I found this job listing [0] under "Computer Gigs" sidebar for the Dallas Craigslist. I suspect that there are similar opportunities all over the country.

      [0] https://www.craigslist.org/view/d/dallas-dallas-redditors-ge...

      Reddit is done. I'm so glad I got perma-banned by their AI mod. Though the meat-space mods lifted the ban on appeal, I had a long enough interval that I unwound all engagement. I found reddit right after they launched and have spent a lot of time over the last 20 years on various subs. That's all over now.

    • ChrisRR

      Because it's where all the interesting discussion happens once you unsubscribe from the default subs

    • isatty

      It’s still excellent for porn.

  • TuxedoBuddy

    I hate to be that guy but this is apparently an HN trope that you’re describing: https://news.ycombinator.com/item?id=1524184

grounder

I thought they used Signal now /s

ovnz

So it is not that classified after all? /s

beedeebeedee

I’ve heard people suggest this is growing pains for the Genesis Mission coming online (we’re hitting the one year mark, and they may be a little cavalier trying to meet their timeline).

reedf1

Anybody know anything about how it's deployed? Nasdaq down 2% on the news?

  • AnimalMuppet

    Well, Nasdaq is down 2%. "On the news", and on every single other piece of news that's out there. Tracing it to any one piece of news is usually pure guesswork.

  • gosub100

    > Anybody know anything about how it's deployed?

    Nice try, foreign hacker

comrade1234

How many IT peolke weee later by doge? For some agencies it was all of them.

  • _verandaguy

    Perhaps the most visible casualty of vibe coding has been predictive typing.

    • TuxedoBuddy

      Spelling mistakes are now a shibboleth

      • _verandaguy

        Sure, but then it's only so long until someone figures out how to prompt Claude without "don't make mistakes." Then it's all over.

    • beerandt

      Pretty sure the pool of people using it (and therefore training it) got a lot dumber.

      If more people misspell than not, then what do the predictive statistics result in?

      It peaked at early mature swype, and has been downhill since then.

  • flowardnut

    dawg are you driving and texting

postsantum

Source:

> according to six people

  • g-b-r

    I don't think people leaking infos about a highly classified network would want to have their names published

  • gilrain

    six people... and CNN's vetting of them, resulting in CNN deciding to publish

    • bhhaskin

      Wasn't it CNN who published that the national weather service's radar was down the other day when that wasn't true?

      • throw10920

        It was. They've been on a streak of journalistic malpractice recently - including an egregious example a few years ago where they slandered and doxxed a minor.

      • postsantum

        The same CNN that is currently peddling another WMD accoring to some super-classified intel

empath75

Once is happenstance. Twice is coincidence. The third time it's enemy action.

  • OvervCW

    Enemy action or incompetence?

    • baggachipz

      Hanlon's Razor seems very appropriate here.

      • Timon3

        Okay, but let's ask why there might have been missing knowledge/competencies/people, leading to "stupidity": the current US government has deliberately massively decreased the value of positions in public service. It's only reasonable to assume that the best people left, considering their job security was more reliant on the opinion of unqualified "hackers" and ChatGPT than on their own job performance.

    • apercu

      Right? These are the same companies that laid off their tech staff because "AI".

  • gpvos

    When did this happen to the NSA before? (No criticism, I just don't know.)

  • whimsicalism

    Once is happenstance, twice is coincidence. the third time is the media has realized that outage stories are getting lots of eyeballs right now.

  • temp0826

    Keep seeing this movie quote like it's something serious. No, this is just more stupidity at the root.

catigula

We need more transparency ASAP or this should essentially be treated as a serious AI related event.

LastTrain

"That's life"

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection