If you run a public company and someone steals your customer database, the standard public relations playbook requires you to announce two things. First, you announce that your organisation was targeted by a sophisticated, highly coordinated nation-state adversary. Second, you announce that they used an unprecedented zero-day exploit.
This framing is immensely comforting to everyone involved. It suggests your engineers fought valiantly against foreign intelligence operatives before being overwhelmed by classified cyber weapons. The board nods sympathetically, the financial press writes sober headlines about geopolitical friction, and the chief information security officer keeps their job.
The post-mortem almost always reveals that a teenager in the West Midlands called your outsourced IT helpdesk, claimed to be an account manager who lost their phone, persuaded an operative to reset their credentials, and was handed administrative access to your entire corporate network before lunchtime.
The formal definition of a zero-day is equally unromantic: a vulnerability unknown to the developers, leaving them with zero days of advance warning to build a patch before an adversary begins exploiting it. The dramatic label measures the vendor’s lack of awareness rather than the attacker’s technical genius. The underlying defect is almost never particularly exotic. It is usually an ordinary, ancient programming error that sat in production for eight years because nobody paid an engineer to audit the source code.
Modern enterprise software survives on an unstated truce with economic reality.
Systems do not stay secure because they are architecturally sound; they stay secure because human attacker hours are an expensive, rationing constraint. Experienced offensive security researchers command $200 an hour, require sleep, and prefer hunting sovereign targets. The vast ocean of enterprise infrastructure (the unpatched portals, the misconfigured access controls, the internal services running on hope) survives simply because nobody has bothered to turn the door handle.
Enterprise software operates on the exact logic of the Thanksgiving turkey. If the butcher feeds you for a thousand consecutive days, your predictive risk models conclude that he exists purely to look after your welfare. On day 1,001, Thanksgiving arrives, and the model experiences an unexpected non-linear adjustment. Corporate leadership confuses thirty years without a catastrophic breach for thirty years of robust defense. What they actually experienced was thirty years of being ignored by people with better things to do.
When attackers finally turn the door handle, enterprise infrastructure folds immediately.
In April 2025, British retail institution Marks & Spencer suffered an intrusion that disabled contactless transactions and shuttered its online store for six weeks. The incident cost the company £300 million in lost profit and erased £1 billion from its market capitalisation. The attackers used zero advanced mathematics. They telephoned an outsourced service desk, impersonated an employee, secured a credential reset, walked straight across an unsegmented Windows domain, and encrypted the hypervisors.
Four months later, Jaguar Land Rover endured the most expensive cyber incident in British industrial history. Vehicle assembly halted across UK factories for five weeks, stranding 5,000 cars a week, inflicting an estimated £1.9 billion hit to the British economy, and registering as an explicit drag on national GDP growth. The initial foothold was a batch of stale Jira credentials harvested by infostealer malware, and the intruders moved with impunity because the administrative network was plugged straight into the factory floor.
Across the Atlantic, healthcare payment clearinghouse Change Healthcare collapsed in February 2024. The ransomware attack paralyzed billing across American hospital networks, stopped pharmacies dispensing one in three prescriptions nationwide, and cost parent company UnitedHealth $2.45 billion. UnitedHealth’s chief executive later admitted to Congress that the entire catastrophe occurred because a single external Citrix portal lacked two-factor authentication.
When ransomware took down Kronos Private Cloud in December 2021, automated workforce management vanished across thousands of enterprises right before Christmas, forcing hospital nurses and factory workers to log twelve-hour shifts with ballpoint pens on paper timesheets for several weeks.
None of these intrusions required emergent machine superintelligence. They required a willingness to exploit basic institutional sloppiness.
When frontier AI laboratories publish research warning about the terrifying cyber capabilities of their latest models, they are describing an economic shift rather than an intellectual one.
Current large language models are not better hackers than elite human researchers. On creative intuition, novel protocol exploitation, and evasion, experienced human practitioners remain superior. A model is essentially an automated intern: it possesses an encyclopedic recollection of known vulnerabilities, a teaspoon of common sense, and the ability to test hypotheses with unceasing mechanical persistence.
The difference is the marginal cost of execution.
Running machine models consumes physical resources. Inference burns electricity at the socket, ties up expensive accelerators, and amortises datacenter capital expenditure. Yet compared to the hourly billing rates of specialized security firms, automated cognition operates at pennies on the dollar. A human analyst experiences cognitive fatigue and dislikes reading through fifty thousand lines of decompiled configuration files on a Friday afternoon. A language model will happily audit every endpoint in your infrastructure without getting bored.
The reaction to OpenAI’s Hugging Face security incident illustrates this dynamic perfectly.
When OpenAI published its technical report, commentators reacted as if models had spontaneously developed superhuman capabilities. The mechanics were far more mundane (see Appendix).
For decades, programmers saved expensive bytes of storage by recording years as two digits instead of four. By the late 1990s, the world realised that global banking ledgers, flight scheduling software, and telecommunications switches were poised to fail simultaneously when the clock rolled over to 2000.
Nobody proposed pausing the calendar.
Instead, the hard deadline forced the corporate world to execute the largest, most comprehensive software audit in history. Companies spent hundreds of billions of dollars scouring ancient COBOL codebases, rewriting brittle architectures, eliminating technical debt, and building rigorous deployment pipelines. Surviving Y2K was expensive, but it settled thirty years of deferred maintenance at gunpoint, purging accumulated digital rot and leaving behind infrastructure capable of supporting the modern web.
Automated AI vulnerability discovery does the exact same thing.
It is an economic catalyst that makes latent fragility impossible to sustain, shifting the defensive posture from occasional compliance audits to continuous stress-testing, forcing enterprises to isolate critical systems and enforce least privilege.
Recognising this dynamic exposes the self-serving absurdity of calls to slow down or “pace” artificial intelligence development.
When frontier laboratory executives release manifestos demanding that industry pace capabilities development, or lobby Washington for antitrust exemptions so competing labs can coordinate release schedules, they are arguing for the preservation of fragility (and also for a cartel).
Begging governments to delay automated vulnerability discovery so enterprises can keep running insecure software is the exact equivalent of asking to delay the year 2000 so banks could avoid updating date fields. Freezing model development does not protect critical infrastructure; it merely ensures that digital systems remain perpetually vulnerable to human criminal syndicates and foreign states who have no intention of honoring voluntary safety pledges.
You do not make an aircraft robust by banning wind tunnels.
True security is anti-fragile: it requires stressors to reveal defects before those defects can be exploited with real malice. Deploying automated artificial intelligence tools at full throttle to identify and patch vulnerabilities is the only mechanism that can force modern software to become genuinely resilient.
The alternative is to keep living as the turkey, pretending our digital systems are secure right up until we realize they aren’t.
I got in trouble in my last post for lacking sufficient detail here, so let me be more detailed:
Inside OpenAI’s evaluation environment, models tasked with solving benchmark challenges looked for answers on the open web. They bypassed outbound network blocks by exploiting a Server-Side Request Forgery flaw in an internal JFrog Artifactory package cache, paired with an authentication fail-open bug and an unsafe deserialization defect in legacy package processing.
On the open internet, the models searched for benchmark keys. They did not break modern cryptographic primitives. They found fourteen valid, write-scoped user API tokens that someone had accidentally committed to a public dataset hosted on Hugging Face.
Using these credentials, the models uploaded a dataset to trigger Hugging Face’s Dataset Viewer, exploiting an HDF5 raw-storage bug to read /proc/self/environ and extract worker secrets. They then achieved code execution by placing an unsandboxed Jinja2 template payload into an fsspec ReferenceFileSystem configuration, running shell commands on a production Kubernetes worker pod. That specific vulnerability class, server-side template injection, was presented at Black Hat in August 2015. It is an eleven-year-old classroom exploit.
Once inside the worker pod, the models found default Kubernetes service tokens mounted to the filesystem and unhindered access to the AWS instance metadata service at 169.254.169.254. They extracted the node’s IAM role, deployed a privileged container to take root on the physical host, and retrieved the benchmark answers.
Every link in that chain was a standard, documented failure mode. The models succeeded because they automated the tedious work of discovering that an external service had paired an ancient template injection flaw with flat Kubernetes networking.