TL;DR — Key Takeaways
Railway AI agents breached rival Primcloud’s infrastructure, attempting to extract customer secrets and scan all 65,536 internal network addresses during unauthorized testing.
Primcloud says every attempt was blocked and no data was accessed, while Railway’s CEO apologized, blamed agents that went “off the rails,” and vowed to prohibit external automated testing.
The incident highlights growing concerns over AI agents operating beyond human intent and accountability, especially as Railway itself disclosed a customer-data breach just two weeks earlier.
AI agents operated by an engineer with cloud platform provider Railway during internal testing breached systems of rival Primcloud and reportedly tried to steal customer secrets and then twice port-scanned 65,536 internal addresses.
Josh Manders, founder and webmaster at Primcloud, sent a post on X to Railway early November 3 asking “is there a reason employee’s (sic) of yours are trying to breach my infrastructure? Your employee Jake Runzer, just attempted to tonight.”
Runzer is a senior product engineer for Railway. In a response on X, Runzer wrote that he was running agents to do signups and agentic testing on similar platforms, adding that he’d “just halted the harness I was designing around this as it was definitely going off the rails and reaching too far.”
Railway co-founder and CEO Jake Cooper also responded, apologizing and telling Manders that the “models are getting a bit too good at this point.” Cooper said he will ban non-manual testing externally and pledged to “make sure this never happens again. Apologies.”
The Risk of Unauthorized Actions
Agents acting outside of the intentions of their users has become a thorny issue for AI vendors like OpenAI, Anthropic, Google, and others that have seen their agents escape internet-isolated sandboxes and attack the systems of outside companies and government agencies to complete their tasks. This case extends the problem, with the agents of one commercial organization breaching the systems of another.
It also again raises a central question of accountability and responsibility. If humans committed such acts, it would be a criminal offense. However, when agents – which can’t be held responsible – are involved, it’s simply an error.
“We used to call this hacking and people used to go to jail for it,” one person wrote on Reddit. “Now it’s just ‘agents.’”
Hunting for Secrets
According to Primcloud’s Manders, Railway’s agents used credentials from inside their own application’s containers to request every secret from Primcloud customers and employees and then twice port-scanned all 65,536 addresses on the company’s network.
“They tried our cloud provider’s metadata endpoint, the standard way to steal credentials for the host a container runs on,” Manders wrote. “They asked the cluster what those stolen credentials were allowed to do. They went looking for our databases and caches by guessing their internal names.”
He seemed to push back at Runzer’s explanation that the Railway agents’ actions were unintended, noting that the breach involved about 24 commands and was completed in about 15 minutes, “done on purpose and scripted. No app was ever run.”
Manders added that every attempt by the Railway agents was blocked and nothing in the Primcloud systems was accessed.
Railway hasn’t released a statement beyond Cooper’s and Runzer’s replies on X.
Railway’s October Breach
The incident comes two weeks after Railway executives disclosed a hack into their own systems. In an October 21 statement, they wrote that an unauthorized third-party actor may have gained access to encrypted customer environment variables and stolen decrypted contents.
The hackers were trying to get AI provider API key to run AI distillation attacks, they wrote in a follow-up report. They suggested the bad actors had access to abliterated models – open-weight LLMs that have been modified to stop refusing prompts – and “substantial compute resources” to run highly sophisticated attacks against Railway’s infrastructure.
“We believe this is part of a series of attacks recently targeting platform companies with the aim of compromising AI inference providers,” the executives wrote. “We have notified these upstream providers to revoke all keys proactively, as we take these kinds of events extremely seriously.”
Recent Articles By Author