Ask HN: What gives Cloudflare the right to takedown apps revealing site real IP?
I stumbled across an interesting app called “CrimeFlare” and what it does is reveal the real IP website’s using Cloudflare’s Ddos Mitigation Service.
CloudFlare had it taken down. https://github.com/zidansec/CrimeFlare
I’m assuming it does this by scanning the public internet in it’s entirely, indexing the domains. (A household fiber connection can scan the entire IPv4 space in a mere matter of weeks)
This is obviously a huge threat to CloudFlare’s entire business model and it totally makes sense that they want to bury this.
I just fail to understand what grounds they have to take something like this down. Internet IPs are public knowledge and these websites are publicly accessible. Just because Cloudflare built a billion dollar buisness exploiting the fact that sites “real” IPs can be hidden through obscurity, doesn’t mean they should be able to censor/takedown apps that expose the flaw in their business plan!
Anyways, I intend to create a new internet-wide scanning system in order to revive the functionality of CrimeFlare just to prove a point that security through obscurity is no security and all, and that CloudFlare doesn’t have the right to take something like this down! > CloudFlare had it taken down. I'm not sure where the idea that we took this down came from, but I checked with legal and we didn't. Such tools, services, etc. have existed forever. Just one reason why we encourage people to protect their public IP (https://developers.cloudflare.com/fundamentals/get-started/s...) and have Cloudflare Tunnel (https://developers.cloudflare.com/cloudflare-one/connections...). Thanks for clarifying that it had to be Github. The post you replied to says Gitbub or Cloudflare take it down. Either way, this issue should be brought to customers attention more clearly. Most people probably don’t know that the entire internet can be scanned in a matter of hours or days which might uncover their site. I’m curious how many customers are paying for your anti-ddos service yet their sites are easily findable using such a tool effectively rendering the service useless. Do you scan the internet yourself and proactively warn customers when their real IP is findable in this way? > Do you scan the internet yourself and proactively warn customers when their real IP is findable in this way? There is no reason for them to scan the internet. They could simply probe the configured origin server from an IP outside the whitelisted cloudflare IP range, and display a warning if it's accessible. Say that, despite your linked recommendations for hiding the public IP, thousands of customers were under the impression that as long as no one leaked the IP, no one would be able to discover the site. They’re paying you a lot of money for security, yet that security can be completely undermined by a teen with a scanner tool. If there’s thousands of clients paying for anti-DDOS services yet their IP is easily findable, then it’s like…what are they even paying for? On a scale of thousands this probably adds up to a large sum of money…Money paid for pointless services rendered. As someone on the “buy side” of Cloudflare-like services, that’s not how it works. How could a third party like Cloudflare protect my unprotected IP address? A very basic part of using a CDN/DDOS protection product is not allowing raw traffic to your origin server. RE “as long as no one leaked their IP” - the IPv4 space is quite small. It’s trivial to scan it and discuss unadvertised, but ultimately very public, servers. If customers don’t already have an understanding of both of these points, then they need to increase their competence in areas that are, frankly, pretty basic. > How could a third party like Cloudflare protect my unprotected IP address? Simple, they could scan the internet like I explained and notify their customers who’s site IP is findable this way with a big scary warning message. They could do this easily and cheaply, but for some reason they don’t. Well they wouldn’t need to do that, because you’re already pointing them at your IP, right? Cloudflare are providing the service they say they are, it’s the customer’s fault if they don’t understand basic best practice. Security tools, when misused or misunderstood, may have security weaknesses. My house has a lock on the front door. Yet that security can be completely undermined if a teen throws a brick at my window. That isn't the fault of the manufacturer of the lock on my front door. > This is obviously a huge threat to CloudFlare’s entire business model I disagree. There are plenty of ways to hide your origin server, for example: 1. IPv6 only, since there are too many addresses to scan 2. Accepting connections only from cloudflare IPs (probably not enough on its own, since features like workers might allow an attacker to trigger requests from a cloudflare server) 3. Mutual TLS authentication 4. Authentication headers (since mTLS might be difficult to integrate in your application) 5. Responding only if the right host is requested, which could even be different from the public domain (not enough on its own, but prevents untargeted scans) 6. Using tunnels (as frizlab pointed out) I think cloudflare already supports all of these out of the box. They just need to push their customers to apply such mitigations via documentation, displaying warnings if the origin server can be accessed directly, etc. So I consider this an inconvenience for cloudflare, but not a huge threat. They have tunnels now. The source does not have to be open to the public at all anymore (the tunnel is a kind of VPN between the source and Cloudflare; all the source has to do is install a single binary) Cloudflare can easily do 4 as well. Use Transform Rules to inject Authorization header or any other one you want. Why on earth would you try to help DDOS'ers? I think you should really take a step back here and reevaluate what drives you here and what impact you have on other people. Publishing such tools raises awareness of the weakness, and pushes vulnerable origin servers to fix it. Ideally cloudflare would show a warning in their UI when the origin server is publicly accessible. There is a website currently publishing my (outdated) informations without my consent (old home address, current email, old phone number) and it is hiding behind cloudflare. I wrote to cloudflare months ago, and silence... So there can be many sides to that story here... edit : oh and what the hell, name and shame https://www.reversecanada.com/ (and they have variants for other countries) aren’t there any legitimate use-cases for it? Yes, for example, pirate websites are often hiding their identity and if someone is infringing on your copyright you can't go and report it to their hosts because Cloudflare hides the IP. Reporting DMCA to Cloudflare won't give you the IP of their hosts. A court ruling exempted Cloudflare from its users infringements of copyright making things easy for them. The said ruling: https://arstechnica.com/tech-policy/2021/10/cloudflare-doesn... In practice however, when you peel off Cloudflare you'll be stumped anyways as the DMCA request will be plainly ignored by those "bulletproof" hosts, so I don't think that knowing the real IP would change your chances, and if you're formally filing a case why just not subpoena Cloudflare? Before CloudFlare sends the FBI to my house..I’m not actually going to code this. It’s just an idea that exposes a problem. The problem is there’s a lot of Cloudflare customers who don’t have their servers configured properly to defend from it. If my amateur self can conceptualize this idea it means cybercriminals already have similar tools and are using them already so If you’re a site operator you should use this post as a warning and fix your servers ahead of time. However, it was messed up they might try to take down the tool rather then help mitigate the flaw. Technically speaking, GitHub took the repo down. This is an important distinction, because voluntary takedowns and legally compelled takedowns are two entirely different things, and it’s not necessarily correct to assume the latter. > This is obviously a huge threat to CloudFlare’s entire business model and it totally makes sense that they want to bury this. Protecting origin servers is hard. Nothing unique to CloudFlare about that. If you follow their set up documentation then this tool can't harm you: https://developers.cloudflare.com/fundamentals/get-started/t... If folks are really concerned about getting exposed they can firewall off everyone except cloudflare. Or better yet: use Cloudflare Tunnel to connect your origin to Cloudflare without exposing any inbound ports. I think you can also have Cloudflare present a client certificate that you can verify before responding. Authenticated origin pulls are mutually exclusive with their tunnel. If you configure your firewall so that only the cloudflared tunnel process can access your origin server, then you can already be assured the request is coming from Cloudflare. >, I intend to create a new internet-wide scanning system in order to revive the functionality of CrimeFlare just to prove a point that security through obscurity is no security and all, I'm not familiar with CrimeFlare and its technical details but a cursory google search shows that security-through-obscurity is possible with Cloudflare if one follows the correct sequence of steps to hide the ip. Otherwise, a careless setup such as public MX mail record will inadvertently "leak" the ip. E.g. Stackoverflow Q&A:
https://stackoverflow.com/questions/58591448/how-does-crimef... >, I intend to create a new internet-wide scanning system But the host systems at the receiving end of your scanning tool still have to respond to your tool pinging them with network requests and if your ip origin isn't Cloudflare, the host server doesn't have to reply with useful information. Or did you have another mass scanning technique we're overlooking? What % of Cloudflare customers actually have their server set to only accept traffic from cloudflare IPs? Probably not the majority. If this is coming as a surprise to people then maybe Cloudflare isn’t doing enough to help people secure themselves against it. As someone else (https://news.ycombinator.com/item?id=31096321) pointed out, everyone does that already, with Shodan (https://www.shodan.io/) being one of the most popular ones. I don’t trust Shodan (a Corporation) not to hide / omit certain results or certain ranges. A self hosted scanner that could be deployed on a cheap VPS would be a better solution. having the ability to scan the entire internet is pretty fascinating honestly. Who knows what kind of sick and bizarre content dredging the entire internet with no filters might dig up. I genuinely don't get your point to be honest. First, you allege that Cloudflare took down a repository that you claim might harm their interests. It could be indeed Cloudflare or it could instead be just GitHub noticing this repository by the "crime" in the repository name. Second, you have said that one legitimate reason is because "they categorically block" Tor (https://news.ycombinator.com/item?id=31095920). I asked you if you can give a website that is fronted by Cloudflare and has blocked Tor users and is otherwise not something that would usually block Tor users and VPNs in any situation like banks (https://news.ycombinator.com/item?id=31095982). I haven't received any reply from you or even someone else to substantiate this. Then, you said that you will build a similar too to scan the IPv4 internet space. Guess what: automated nefarious scanners are pounding on every IPv4 address to find unprotected systems, either because it's Windows and it's SMB feature is so bug-ridden that exploits are patched nearly every Patch Tuesday. Or old Wordpress installations where fully-automatic worms will hijack the site for spam links. Or even directly hacking routers and servers for botnets. It already exists to be honest, so I don't get why are you pretending that this is a new vulnerability or something. It’s a story in itself that a simple script which locates a site’s real IP was taken down for TOS violations. Cloudflare doesn’t own the real IPs or something so it’s really unclear why they (or GitHub) were entitled to take down this repository. Just because it threatens their million dollar buisness model they think they can take it down? That’s wrong my friend. And people need to know. Cloudflare or GitHub overstepped it’s boundaries to help a corporation enforce security by obscurity. Since this method is proven to be preventable, why take it down? Instead of taking it down from public knowledge (which does nothing to stop cybercriminals with private forks) why don’t they help their customers mitigate the impact instead? Also, they stopped blocking the Tor IPs now but this wasn’t always the case. Many people remember a few years ago the IPs were blocked. As I stated above (https://news.ycombinator.com/item?id=31097086), we did not take this down. Please stop claiming that. Apparently you will not believe anything, but the CEO responded here: https://news.ycombinator.com/item?id=31097086 Scanning the internet and indexing domains? Isn't that EXACTLY what binary edge and shodan do??? If you are going to use someone else to front your service, take care to make sure that that (1) it cant even be accessed except via that front, and (2) that you dont leak your origin IP address or network, even if traffic to that origin is dropped from sources other than the service fronting it. How can you index domains by scanning the public internet? Wouldn’t trying to match domain names with IP addresses get you blocked by the server after too many failures? Or at least it would be too many attempts to make that it would take more than weeks? >by scanning the public internet in it’s entirely, indexing the domains Can you explain this? So there’s only 4.2 billion possible IPv4 addresses where a site can live. A lot are reserved or unused, leaving about 3.7 billion possibilities. Household internet speeds are fast enough that it is within the realm of possibility that a computer could sequentially connect to every single IPv4 host on the entire internet in search for the target website. Specialty network cards with datacenter connections can scan the entire Ipv4 space in a matter of mere hours. If the origin server only responds with the relevant content if the correct host is requested, this scan becomes much more expensive, since you need to scan all IPs for each domain you're interested in, instead of once total. But what makes you think that the real IP serves the same site to the public internet, as it proxies to CF? If I were using CF for DDoS mitigation, I would drop all traffic to my real IP other than traffic originating from CF. Your approach would definitely protect you. In practice, many site owners don't do this, or they configure their web server with the whitelist instead of their firewall, denying direct access but exposing information about their domain. For site owners who don't know about this, these are the IP addresses you can expect traffic from: https://www.cloudflare.com/ips/ I'd personally advice using IPv6 (with a high, random address rather than the common aa:bb:cc:dd::0) to make scanning for hosts a lot harder to accomplish, just in case your firewall fails for some weird reason. Thats the correct way of handling it, problem is not that many sites actually do that, or atleast they didn't used to. Back in the day before teespring had a public API I was scraping order counts from product listings, prob was the main domain was behind CF so the "sold count" was always cached and I wanted the live number. I actually used "CrimeFlare" back then to get the real IP of the origin server and queried that instead. And thats TeeSpring. Twitch also until recently had their origin server open to all (allthough it would often bounce you back to www.) The name might be infringement or the code might abuse their API. Or, GitHub could decide it's not worth it. Why would you try to scan every IP address? A valid use case for wanting to know the “real” IP of a site hiding behind CloudFlare is being able to access the website from a Tor IP address (which they categorically block). For users in a country with censored internet, such a service would be essential. > which they categorically block Everytime I check this statement with Cloudflare-enabled sites... it was either always accessible (a nagging screen might be shown momentarily, but that's it), or the block is usually due to that site being a bank or something else that will block Tor users regardless of their firewall solutions. I've just tested it again just in case something has changed, but that statement holds up every time. Can you please give a non-banking site that a) uses Cloudflare and b) blocks Tor? They stopped blocking them now but used to in the past. Additionally, there are privacy reasons a person may wish to access a service directly and not be tracked by Cloudflare. Cloudflare doesn't automatically block Tor exit nodes, they just tend to earn a bad reputation. As a site owner you can decide what to do with that, the basic protection level issues CAPTCHA challenges. Cloudflare also has it's own onion service, sites can opt in, and Cloudflare's public DNS is also available over it, sidestepping the need to go over exit nodes after the first request. If your website is behind CloudFlare, why even allow direct connections from anyone that's not CloudFlare? With Cloudflare's tunnels, there's no longer even a need to allow direct connections from the outside world. In my own testing it wasn't too terrible to setup firewall rules and mutual TLS-based authentication of origin pulls, but it is certainly something where you have to do everything right to be as secure as you think you are. Versus just closing off inbound connections entirely and running cloudflared that’s just the client though it only makes request to https://api.xploit.my.id/v1/crimeflare.php and logs the output Nope, that's literally it: https://web.archive.org/web/20210426093141/https://github.co... It appears zidansec runs xploit.my.id. The api subdomain is not currently responding. https://www.xploit.my.id/2021/07/crimeflare-bypass-tools-clo... as far as i remember when the backend times out, CloudFlare shows a screen where you can see the actual IP of the server I have seen the screen where Cloudflare cannot contact the origin and it absolutely does not include the IP address or other details about the origin. You might be thinking of the "Ray ID" that Cloudflare displays on that page, which is just a random request ID that has nothing to do with the origin server. That would defeat the whole purpose of using Cloudflare as an anti-ddos measure so I doubt that. Do you even know under which rule it gotten taken down? Just another reason to add to the pile of why I hate that company. They probably reported it as malware and M$ team didn't check what it was
^E^T