Settings

Theme

Cloudflare AKE cuts origin HelloRetryRequests from 52% to 3.7%

blog.cloudflare.com

67 points by iamsyr · 22 comments

Reader

5 threads
sandeepkd

TLDR;

1. The TLS handshake involves a step to discover the commonly supported algorithms and can incur additional roundtrip if the first guess does not works out, its part of the protocol to keep it stateless

2. Cloudflare is scanning all the origins on daily basis and storing the result for supported algorithms to save on the possible roundtrip time

Whats missing in the article - They are saving on the *possible roundtrip latency, however they are not sharing the absolute lookup latency which now gets added to every connection

  • sophacles

    What latency gets added? Presumably on cache miss they already need to look up "whats the origin for www.example.com" and get info about it. This is just a handful of bytes in that record.

    • sandeepkd

      They have to check from their data store on the algo they scavenged for the clients, thats the whole idea about being correct instead of making a guess.

chrismorgan

Genuine question: why wouldn’t they have been doing this already? It feels like obvious low-hanging fruit on a critical path, so I presume there’s something more to it than I’m imagining.

  • edelbitter

    Speculation, no insider info: Its one more thing that needs to be sent out to all their proxies, and could be a reason for meaningfully different metrics whenever the (non-time-critical) regular updates fail. The more configuration/state each proxy receives and processes, the more difficult it will be to determine what is wrong when one exhibits abnormal behavior. At least one of the serious outages was even directly attributed to having accidentally exceeded some hard limit (very hard actually, a rust panic) in how much data could be distributed through one particular such channel, see https://blog.cloudflare.com/18-november-2025-outage/

greatgib

2 things comes to my mind reading this article:

1) So they saved 15ms on the connection so that you can then wait 20s in their annoying nag screen before reaching the real website content.

2) On the Monday they complain about the load on server by LLM scrapings compulsively your webserver an offer themselves as the internet guardian solution; and on Tuesday, they compulsively send useless requests to your servers so that they can save a few microseconds in the very first connection ever to your server. "For each TLS 1.3 capable origin, we run a series of a few lightweight TLS handshakes, each offering exactly one key agreement group: X25519, P-256, P-384, P-521, or X25519MLKEM768. [...] And because the active scanning happens outside your production traffic path, we confirm that both your origin and the network in between can handle connections with a stronger key agreement before any real traffic depends on it.

  • innocent_name

    >they compulsively send useless requests

    Those requests aren't useless; They clearly optimize. What a silly take.

    >to your servers

    To their customer's servers, right! Most people turn on other CF optimizations like h2/h3 to origin.

  • sophacles

    2) Perhaps theres a slight difference between thousands of requests per day from untrusted entities and a single TLS handshake per day from a trusted one? (Note i understand you clearly don't trust cloudflare, but the people who sign up for cloudflare do trust them - that's the trust relationship i refer to here.)

  • gonzalohm

    Agree on point 1. Now even tiny websites that wouldn't be a target for anyone are hiding behind Cloudfare. If you are so worried about people sending requests to your website then take it offline, that will get you 100% success preventing bots

    I'm so tired of having to do Captchas and waiting everywhere to access websites

    • Kodiack

      As someone who hosts a few “tiny websites” that are “hiding behind Cloudflare”, it’s because of some incredibly misbehaved botnet traffic that’s otherwise persistently scraping via residential proxies.

      This is a hobby for me. It’s for my enjoyment and it allows me to provide resources that others enjoy using. However, I’m not going to allow literally 99%+ of requests to be aggressive scrapers that won’t give up until you’ve got a heavy-handed solution in place.

      I hate it too, but the alternative is even more consolidation, so unfortunately this is just the reality right now and you’ll have to get over it until when/if things improve.

      • tredre3

        > I hate it too, but the alternative is even more consolidation, so unfortunately this is just the reality right now and you’ll have to get over it until when/if things improve.

        No, the actual alternative is learning to do it yourself. Unless you're a frequent target of UDP DDoS, there is almost nothing you can't defend against straight from the server itself.

        Spending time learning how to configure your firewall, set rate limits in your web server, tune your application for caching and further rate limits, and should all this fail install a self-hosted captcha/proof-of-work fence yourself, is definitely a fair amount of efforts when those things aren't part of the core hobby.

        • karlshea

          Sorry, this just isn’t true. Some clients just don’t have the resources to pay for a dev playing whack-a-mole against residential proxy bots that are basically indistinguishable from legit users, but en masse can take down a VPS.

          Rate limiting doesn’t work when individual IPs are only making a handful of requests.

          So that’s why we’re using CF/Fastly. Too bad, I’d prefer not to, but there is just not an alternative at this point.

ttul

Cloudflare is the canonical example of why you can't vibe-code infrastructure. Knowing that this optimization was even necessary, let-alone having the ability to build it, is something that doesn't become apparent until you're operating at considerable scale. Once there, of course if you're Cloudflare, you use coding agents to build it. But outside of these temples of scale, good luck even knowing it was needed.

If you work at a SaaS of any kind, I think it's worthwhile considering what things will look like when scale is the only thing that is really defensible anymore.

  • asdfman123

    The irony is that everything about this speaks to vibe coding. The writeup was written by Claude, in a good way (E.g "while the milliseconds are important, that second part may matter more").

    I think things like this are now possible through vibecoding.

    • binsquare

      I like to think that the coding agents has basically raised the bar.

      Those who are above the bar can steer and add their expertise to hit a new level.

  • doctorpangloss

    Brother, if you think Cloudflare isn't vibe coding features...

    • ttul

      That's not what I'm saying at all. I guarantee Cloudflare engineers are using coding agents every day for everything they do. The point is that discovering what you need to build comes only with the experience of operating at tremendous scale. I do not believe that someone could replicate what Cloudflare offers today just because they, say, have invested $20K/mo for 100x Claude Max plans, all of which are cranking 24x7.

      • doctorpangloss

        nothing stops you from simulating load. something that scales, by definition, it will have something in the small that is more or less identical to something in the large. it goes both ways.

        the obstacle was always the parade of tedious obstacles to create (in this case) a sophisticated virtual harness to simulate (in this case) whatever the hell this article is about. so yeah, actually, the capability of retail coding plans are kind of the only thing that matters...

        • immibis2

          To even simulate the kind of load Cloudflare gets, you already need to be Cloudflare.

lucaprata

We had a probe that checked whether a connection was using post-quantum cryptography. It looked for "Cipher is" in the output. But when the handshake failed, it would print "Cipher is (NONE)." So it reported that github.com and amazon.com were compliant. For weeks.

The success criterion was embedded in the errorline.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection