Settings

Theme

How to stop a DDoS attack

blog.fastmail.com

114 points by robn_fastmail 10 years ago · 28 comments

Reader

DanBlake 10 years ago

A article I wrote on attempting to do this yourself (without cloudflare/other services) is here: http://harknesslabs.com/post/38104429912/fighting-spoofed-sy...

Its much easier to use cloudflare, but sometimes it just not possible to use them (it wasnt for us, due to needing hardcoded IPs in our DC)

  • pki 10 years ago

    Cloudflare lets you announce your own IP space through their serivce on non-personal plans don't they?

    • vox_mollis 10 years ago

      Not that I'm aware of. You're thinking of GRE tunnel DDoS mitigation providers like staminus and blacklotus.

    • asherkin 10 years ago

      You need an Enterprise plan for that unfortunately, it's not even included in Business.

    • rmdoss 10 years ago

      Starts at $5k per month last time I checked.

nullrouted 10 years ago

tl:dr Used cloudflare for DNS and Level3/Blacklotus for network filtering.

In DDoS attacks you have three models: On-Prem: Buy hardware and big fat internet pipes to filter traffic (expensive / time \ resrouce intensive) Hybrid: On-Prem devices that can mitigate X/Mbps and then starts announcing your routes after X to their cloud scrubbing centers which can filter it at a much higher capacity (best option) Cloud: Full on filtering by a provider where all your traffic goes through their scrubbing centers full time (usually adds latency, extremely expensive)

The hybrid model is the best and what most companies are going to as it allows you to filter smaller attacks out with little cost as well as scaling up to large 100 Gb/s+ attacks without having to buy massive amounts of hardware/transit.

  • prdonahue 10 years ago

    How do you define "extremely expensive"? CloudFlare's Business Plan ($200/mo) includes advanced DDoS mitigation: https://www.cloudflare.com/ddos/.

    Also, due to caching of assets in PoPs close to end-users (and TLS termination at the edge), the site is often much faster than without DDoS protection.

    • franimals 10 years ago

      Well CloudFlare works if you're mainly worried about responding to HTTP(S) traffic. In this the company was responsible SMTP, POP, etc which CloudFlare doesn't really handle.

      Additionally as is mentioned in the article - If the attacker knows your public IP address they can easily bypass CloudFlare by simply directing the traffic to you and not CF.

    • nullrouted 10 years ago

      Cloudflare is a WAF/Proxy that can handle DDoS, it isn't a DDoS specific product. If your actual network space is getting hit (e.g. 8.8.8.8) cloudflare will not help you.

      • snowwrestler 10 years ago

        Set your network firewall to drop all packets not originating from Cloudflare's IP blocks?

        • paulfurtado 10 years ago

          Depends on the type of DDoS. Traffic may saturate your internet connection regardless of there being a firewall on your end. In which case you need a provider capable of handling the full bandwidth of the DDoS sitting in front of you.

    • vox_mollis 10 years ago

      How do you define "extremely expensive"?

      Always-on scrubbing will typically run you $10k provisioning and $6-9k monthly for 100mbps of clean bandwidth from most of the providers.

      • nullrouted 10 years ago

        Always on scrubbing has a number of factors depending on the provider you use. All charge for clean traffic but some charge for number of netblocks/ASNs, number of routers, number of attacks mitigated, etc. It can get extremely expensive depending on your traffic needs and all the other variables. Paying an extra $250,000+ year for 2 Gb/s of traffic is extremely expensive to me.

rmdoss 10 years ago

DDoS is becoming an increasing pain lately.

If you only care about HTTP/HTTPS traffic, you can get very solid DDoS protection at cheap prices. We use and love the Sucuri ( https://sucuri.net ) which starts at $9.99 per month.

Some friends have good success with Incapsula and CloudFlare, but they get a bit more expensive to get full protection ($60 per month on http://Incapsula.com ).

All 3 can cover 99.9% of the people that doesn't expose SMTP/POP/FTP/DNS and other services.

If you run these yourself, BlackLotus.com and Arbor Cloud are a great help, but their prices start at 5 digits per month.

dimgl 10 years ago

The irony is that this website seems to be down right now.

http://downforeveryoneorjustme.com/blog.fastmail.com/2015/12...

Not sure if it's due to DDOS, but it's definitely not working on my end.

  • robn_fastmailOP 10 years ago

    Datacentre confirmed a DDoS attack, which took a little while to mitigate. We're looking good now, but we're continuing to monitor.

  • alfiedotwtf 10 years ago

    It's intermittent. Currently works fine for me though:

    http://www.fastmailstatus.com/

  • mbrd 10 years ago

    My Fastmail web access is currently taking 60+ seconds to load. The DDOS in the blog was last month so I'm not sure if it's related.

  • elwell 10 years ago

    Posting a blog post with the title "How to stop a DDoS attack" unfortunately will invite the trolls.

    • robn_fastmailOP 10 years ago

      Quite. As my dear colleague said this morning, "hubris" is the word of the day. Still, I'm not sorry we posted it.

      • gist 10 years ago

        I would honestly like to know the upside to posting vs. the downside. I think there is also a saying for this, something like "don't spit into the wind".

        • robn_fastmailOP 10 years ago

          Generally, we talk about what we're doing because we're all excited about what we do. It's always been that way for us, and our customers really appreciate the honesty and transparency.

          On this particular one, we really did learn a lot and we were keen to share some of that. It's really difficult to run an internet service and we feel we have a duty to try and make this easier, or at less better-understood, where we can.

          Our business can't exist with a large diverse network that anyone can get involved on, and we couldn't have got to this point without the knowledge of others, whether that's embedded in the open-source software we run or in the blog posts and emails of other people that figured out hard stuff. It wouldn't be right for us to take and not give something back in return.

          There's also an element of defiance in this post too. We got punched in the face. We're not going to respond by hiding in a corner. We're going to say "you know what, fuck you" and we're going to help (and have helped) others to do the same in whatever way we can.

andrew_wc_brown 10 years ago

When I was working for a startup that was getting DDOS the only thing that stopped it was this service.

https://www.dosarrest.com/

tracker1 10 years ago

Cool article... how about "Dead or Alive" bounties for the people responsible? I'm only half joking, but given the distribution of the people responsible, and how much like the "old west" attacks on the internet today seem to resemble, not sure how bad of a solution it would actually be.

  • robn_fastmailOP 10 years ago

    I don't know about bounties, because I'm not personally in favour of vigilantism, but I do take your point.

    Honestly, I think the ease in which people can be anonymous is major problem here. Anyone with an internet connection can buy botnet time with Bitcoin and accept a ransom in the same way. It makes it incredibly difficult to follow the path back to the attacker.

    At this point pretty much the only thing you can do is collect data and share it with CERT and other relevant law enforcement. I don't have a good sense of how effective they can be, but it makes sense that the more data they have, the better chance they have at identifying specific botnets and follow the path back to the owners.

NickHaflinger 10 years ago

'A botnet consists of many (usually hundreds or thousands) of normal home or work computers [running Microsoft Windows] that have malicious software installed on them.'

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection