Improving DNS Privacy with Oblivious DoH
blog.cloudflare.comAll: We changed the URL from https://techcrunch.com/2020/12/08/cloudflare-and-apple-desig... to the more detailed source, but you might want to read both.
The biggest and most consistent downside I see with these DNS enhancements is that it prevents filtering at the network level. Querying nameservers is being pushed into applications themselves to support these new features (such as Chrome and Firefox), which bypasses any system resolvers configured on the host. In most cases there is no way to signal from the network that it is not desirable to do this (Firefox being the sole exception). There also is no good way for enterprises to centrally manage these settings. DNS is a major source of information when doing threat hunting on a network and having that go dark is a big problem.
Enterprises aside, there has been a rise of people using solutions like pi-hole in their home networks to filter out traffic not just for ads, but known malicious domains, and telemetry trackers (which Apple does get filtered by, only calling them out specifically because they have an active interest in not being filtered like this).
Yes I think it's also a problem that ISPs are snooping and selling this information, but I think that is a less severe problem than rampant malware infections and the excessive collection of online usage data in the telemetry systems present in every webapp, OS, mobile, or IoT device. This increases privacy in one place, while making it much harder to actively protect yourself from the more aggressive and invasive sources of data collection.
As someone who recently set up a pihole, I was shocked that it was possible to redirect all DNS requests on the network (in plain text!) to the pi. I did the method where you set up a network firewall at the router level that redirects all port 53 traffic to the pi. It's a nice feature for getting my xbox filtered, but it really felt like an insecure historical quirk rather than a feature we should be praising.
Surely it's progress for devices to be able to securely access name servers? I can't snoop on the network traffic going over https but somehow I can get a list of all names queried?
If I get access to those devices, yeah, sure. But in practice, I'd argue this actually reduces privacy for users, as it gives apps and devices a secure path through the network which cannot be monitored by any intermediary - including the user of the app or the owner of the device themselves. So no chance to find out what kind of data is being transmitted there either.
To me, DoH seems less about protecting the user from attackers and more about protecting apps and devices from the user.
I think your point is 100% correct but surely you understand that this applies to HTTPS too. Network filtering would be so much more powerful if my pihole could modify and block HTTP requests in-flight.
One way to limit this would be to add IP resolved through an approved resolver to a temporarily allowlist for a firewall. The firewall would default-deny outbound network requests. Allowlisted IP would be permitted, but be removed from the list after the TTL for the DNS request expires.
Of course, you'd have to add in some permanent exceptions once you realize just how much hardware and software implodes as a result of this.
Some expected, like peer-to-peer applications, though those allow you to define an outbound ephemeral port range you can limit them to (except perhaps for some poor implementations in commercial game launchers trying to offload bandwidth costs). So some are fairly easy to define.
Others you'll have to log and see. Like your Google Home hardware..
I got some minis and disabled the mics since they had hardware switches. I hacked around a bit emulate sending them audio programmatically and discovered they use external DNS and therefore couldn't resolve the local network web server hosting the audio clips I wanted to play. So I had to permanent-lease the hostname's IP and give it an IP address.. They were already bypassing local DNS blockers years ago.
> add IP resolved through an approved resolver to a temporarily allowlist for a firewall
That ship has sailed now that some of the functionality provided by TCP moved up to HTTPS. Whereas in the past you could expect the same IP to expose DNS on port 53, FTP on port 21, or HTTP on port 80, now the same IP will serve you everything over the handy port 443.
So a software developer can very well go this route if they want to obfuscate DNS calls and you wouldn't be able to discriminate the traffic like you would today with ports.
Any global (OS/network) policies become meaningless if your browser or app decide to only ask the DoH resolver "who is google.com" or "who is facebook.com" once, and have all subsequent queries go that way inside an encrypted HTTPS stream.
> cannot be monitored by any intermediary
Key word is any single intermediary. The announcement even explicitly says "no single entity can see both at the same time".
I wonder what it would be like if there were multiple entities in cooperation, each sharing their component of your request with their partners to recover the entirety. You could cover the entire planet with access to even as few as three or five or so large networks.
You're definitely right that it is an issue that the traffic is plaintext but there are trade off costs that are not addressed by these standards, mostly in user control and fall back behaviour.
You've started using a pi-hole, and presumably are getting value from it. These protocols can potentially make it so you can't use that pi-hole at all.
Traffic that is local to a network being unencrypted is not a huge privacy problem. If this protocol was adopted by local resolvers, your pi-hole or network router could use it for any requests it makes while still preserving its ability to filter the traffic. It's basically all win under this scenario.
The problem comes back to applications implementing this in ways that can't be managed taking the option away from end users and administrators. Without the protocols specifying control and fall-back behaviours on networks that don't need or want this, it's more harmful than useful.
Can you explain (or share a link) to some proposal for how to enable my pihole to securely talk to upstream resolvers but force all embedded devices on my network to go through the pihole? Anything that lets my pihole sidestep my ISP seems like it'd also work for my xbox.
> force all embedded devices on my network to go through the pihole
You can only do this for the devices that respect your DHCP-provided DNS config. Even if you redirect all port 53 traffic on your network to your pihole, a device can make its own (DoH or non-DoH) https connection and gets DNS responses via that, bypassing your pi-hole.
This was discussed extensively a few days ago on a thread about "72% of smart TVs and 46% of game consoles hardcode DNS settings":
If you control DHCP and know the MAC address of those embedded devices, you can serve them a non-existing gateway so that they simply won't have a path outside of your home network.
Of course, that assumes IPv4, whereas with IPv6 and SLAAC I believe the only way is to firewall them out.
If they won't ask you to drink a verification can to proceed.
Using Cloudflare with DoH is documented here: https://docs.pi-hole.net/guides/dns-over-https/
You essentially run a little proxy server on your pihole setup, and configure pihole to use it as your upstream dns resolver.
E.g., a proxy server running at 127.0.0.1:5053 which uses the Cloudflare ipv4/ipv6 DNS over HTTPS endpoints. This can also use other DoH endpoints as desired:
/usr/local/bin/cloudflared proxy-dns \ --port 5053 \ --upstream https://1.1.1.1/dns-query \ --upstream https://1.0.0.1/dns-query \ --upstream https://2606:4700:4700::1111/dns-query \ --upstream https://2606:4700:4700::1001/dns-queryThat only does the part where the PiHole uses DoH. It doesn't stop individual devices from using it, and it doesn't force them to go via the PiHole.
DNScurve never got far in the IETF but I like it:
https://tools.ietf.org/html/draft-dempsky-dnscurve-01
It’s being worked on again. The “client side” (from client to resolver) got all the attention in recent years. Even though it’s arguably less important than encrypting resolver to Auth traffic (as the resolver can often be close.).
Cynics say this is because there was money behind DoH which pushed it through standardisation, as the big providers are hungry for the data.
arguably the less important part of the equation
I have used CurveDNS forwarders at home as an experiment for many years now. I have never had any problems. I cannot see why authoritative DNS providers would resist offering DNSCurve as an option. It is relatively easy to set up and does not require replacing or modifying any DNS software.
I don't have any resources handily available for that. I would be truly surprised if pi-hole's don't support DoH though so I'd just try searching for something like "enabling DoH on a pihole" or similar.
Basically sounds like you've already done the hardest parts. Your router is redirecting all DNS traffic to your pi-hole, this will prevent any normal unencrypted DNS traffic from leaving your local network. You pi-hole will be in turn making all the DNS requests. If you turn on DoH for the pi-hole all the DNS requests on your network will be encrypted.
Nothing would prevent DOH to use <randomIP>:<randomPORT> as a resolver. Be it an application or a device. Pi-hole will never see it.
"I can't snoop on the network traffic going over https but somehow I can get a list of all names queried."
SSL/TLS's servername extension puts those names in plaintext just like DNS. The popular browsers all include this extension even when the website does not require it.
As such, one can get a list of names by snooping on HTTPS traffic, instead of DNS traffic:
https://github.com/kontaxis/snidump
ESNI-enabled software and Clouflare's ESNI-enabled CDN is an option, but you have to keep making DNS queries to get a key that changes every hour.
With time, client-side TLS tickets will become more common and session resumption means you won't always be able to see the SNI.
It’s progress if you control your devices, or you don’t control your network.
I don’t. Like most people I can control my network.
I have all sorts of crap on my network from Bose and amazon and Nintendo and Apple etc on my IoT vlan.
Without going to a monk style digital life aka RMS, the best bet is to segment them into a secure network and limit what they can communicate with. The DOH Culture and the like takes away my freedoms.
Then complain about devices that hardcode settings and don't allow changing them, rather than complaining about people taking what devices could already do and standardizing it so that anyone can use it in a more uniform way.
Complain to who exactly?
> people taking what devices could already do and standardizing it so that anyone can use it in a more uniform way.
In this case, standardization makes a huge difference.
Before DoH, this was theoretically possible, but needed enormous effort to pull off: The simplest thing a device could do was to hardcode custom DNS servers - but the network admin could easily bypass that by redirecting the packets, as described in this subthread.
Any more interception-proof solutions would have involved designing a custom network protocol, running a custom server and implementing custom bootstrapping logic to connect the device to the server - and even then, the traffic would stand out enough than an admin could still block or redirect it easily enough.
With DoH, there are publicly accessible servers that accept requests over plain HTTPS: This means, someone who wants to keep their ads from being blocked does not need to run any server infrastructure and does not need to fiddle with network code at all - they can just drop in a DoH client library, hardcode a list of public DoH servers and client certs and be done. This is absolutely a game-changer.
> With DoH, there are publicly accessible servers that accept requests over plain HTTPS
Which is a good thing for end users on balance.
The Internet is going 100% encrypted and that's a good thing. This helps towards that goal. Relying on unencrypted traffic will no longer work. Networks must not have the ability to intercept device traffic unless the device administrator (not the network administrator) configures it as such; any such mechanism completely breaks the security properties this is trying to achieve. Ultimately, your choice with uncooperative devices is "block the device or don't" (in addition to isolating it on a separate network), along with "replace it with a cooperative device". Stop telling people they need to stop using encryption so that your local network interception will keep working, because your local network interception is indistinguishable from other people's local network interception.
This feels like the geek version of DRM fallacies or the fallacies that governments believe about encryption. Somehow, there continue to be threads full of arguments that amount to "It should be possible for 'good' network admins to intercept traffic from devices that don't trust them, but 'bad' network admins shouldn't be able to intercept traffic from devices that don't trust them". That's fundamentally not possible; any mechanism usable by 'good' network admins can be used by 'bad' network admins. (Leaving aside that 'good' and 'bad' are relative.) And worse, people get so invested in their local network interception that once it becomes clear that isn't possible, some will start arguing that their local network interception is more important than general Internet security.
There are enough outside opponents of Internet security; let's not start holding back security ourselves because it makes our own hacks stop working. We have enough work to do fighting for the ability to keep the Internet secure against real adversaries. We absolutely need full control over our own devices, but many of the people most capable of fighting for that control got complacent about it because a subset of people could hack around it with local network interception stunts.
> there continue to be threads full of arguments that amount to "It should be possible for 'good' network admins to intercept traffic from devices that don't trust them, but 'bad' network admins shouldn't be able to intercept traffic from devices that don't trust them"
That's not what I see at all.
I see people pointing out that DoH hurts privacy and reduces control for end users by providing a convenient turnkey solution for device vendors to bypass filtering at the network level.
I also see it pointed out that DoH could have been specified in a way that facilitated filtering for the local network. Given that it's so obviously possible, the fact that it wasn't speaks volumes.
Note that (IIUC) your ISP can still see which sites you visit because TLS still transmits the FQDN in plaintext (https://security.stackexchange.com/questions/86723). Even if that stopped happening tomorrow, the destination IP would still be visible (not quite as bad but still reveals a huge amount of information). On top of all that, DNSSEC already exists which allows you to verify the authenticity of the query result. As such, the argument in favor of DoH would seem to be limited to preventing your DNS resolver (but not your ISP or VPN!) from tracking which sites you visit. I don't find that to be very compelling in light of the immediate downsides.
> I also see it pointed out that DoH could have been specified in a way that facilitated filtering for the local network. Given that it's so obviously possible
No, it couldn't have been, and this is exactly what I was referring to in my comment. Any mechanism that allows the local network to intercept the traffic of a device that doesn't trust the network can and will be abused. The entire point of DoH was to make DNS clients secure, by preventing ISPs and other network providers from monitoring, intercepting, or tampering with DNS results.
You're asking for DNS to be left insecure, so that you can tamper with it. You're asking for the security of clients that actually give users control (laptops, phones, etc) to be sacrificed so that you can continue to tamper with DNS results for clients that don't give users control.
> On top of all that, DNSSEC already exists which allows you to verify the authenticity of the query result.
DNSSEC isn't nearly widespread enough to expect to find it everywhere. Only very specialized clients could make it a requirement; most clients cannot. DNSSEC requires upgrading most of the world before people can rely on it; DoH is an incremental solution.
> As such, the argument in favor of DoH would seem to be limited to preventing your DNS resolver (but not your ISP or VPN!) from tracking which sites you visit.
SNI is being fixed. Once SNI is fixed, DNS is one of the last holes that allows your ISP or other network provider to track you.
And as mentioned above, since DNSSEC is not a viable solution anytime soon, DoH is also critically important to prevent ISPs and other network providers to tamper with your DNS results.
Some color to this: it's less than 2% of North American domains, the number of signed zones has actually dropped in some intervals, and it's practically nonexistent among big companies with security teams. Google isn't DNSSEC-signed. Neither is Microsoft. Or Facebook. Or Amazon (whose DNS service, Route53, doesn't implement DNSSEC). Or, last I checked, any US bank.
You can check this for yourself: make a list of domains, and then write a trivial script:
#!/bin/sh while read domain do ds=$(dig ds $domain +short) echo "$domain $ds" done
I am cynical and maybe paranoid.
Be careful what you wish for
What I think will end up happening, most of the devices will go encrypted route [1]. Most things will run over https, and with encrypted sni, you won't even be able to block domains.
Encryption will be backdoored by governments (and of course other people that will reverse those backdoors or have friends in LEA ).
End result is encryption that people are championing as improving our privacy will make us more exposed and more vulnerable, because other people will have more access to our network, devices and data than we do.
I can and do run linux, so I don't have to worry about my os. But my hardware might phone home on its own. Phones are already lost causes. TV's almost as well.
I do hope you are right and I am wrong, I'd much rater be wrong.
[1] You already have hard time buying non smart TV. A lot of things nowadays expect network connection, and that trend is increasing.
People advocating a 100% encrypted Internet are also many of the same people fighting tooth and nail against backdoored or otherwise broken encryption.
It doesn't make sense to say that encryption will be backdoored so we should use plaintext. We should fight for security across the board, and fight against any threat to that security.
Again, this fight doesn't make any sense if that very same encryption is then used against us.
> This feels like the geek version of DRM fallacies or the fallacies that governments believe about encryption. Somehow, there continue to be threads full of arguments that amount to "It should be possible for 'good' network admins to intercept traffic from devices that don't trust them, but 'bad' network admins shouldn't be able to intercept traffic from devices that don't trust them". That's fundamentally not possible; any mechanism usable by 'good' network admins can be used by 'bad' network admins. (Leaving aside that 'good' and 'bad' are relative.) And worse, people get so invested in their local network interception that once it becomes clear that isn't possible, some will start arguing that their local network interception is more important than general Internet security.
That's not the distinction at all.
It's that you should be able to set up interception on the local network at install time, but all other interception should be blocked. It's extremely possible.
I’m not held hostage by my ISP - I choose one that doesn’t do that. Even if I didn’t trust the last mile, that’s why I can choose to implement a vpn for some or all of my devices on my network so I can bypass a hostile isp.
It might be good for the average technophobic end user to trust google instead of their ISP, but it’s not good for me to trust google over my ISP.
Most users do not have a meaningful choice of ISPs. That was true back in the days of modems, and somewhat true in the days of DSL. It is not true at all in the days of fiber and cable.
> Networks must not have the ability to intercept device traffic unless the device administrator (not the network administrator) configures it as such
The device admin of an iPhone is Apple, the device admin of a non-rooted Android phone is Google, the device admin of a Windows 10 PC is Microsoft and the device admin of a smart TV is likely Samsung. Is that the point you want to make?
All of those companies have incentives to collect user data and are known to push user-hostile changes. They also either have known ties with the NSA or can be coerced via National Security Letters, making even the tired point about "defense against state actors" moot. Do you trust any of them?
If, say, Tencent brought a phone to market in the US that turns out wildly popular but happened to exchange vast amounts of data with servers in China, would you be ok with that?
Most devices today have locked-doen bootloaders, so I don't actually have a choice about the device admin for my device. The position of network admin can be abused, too, but at least there is a possibility I can fill this position myself or can delegate it to someone who I trust.
> Ultimately, your choice with uncooperative devices is "block the device or don't", along with "replace it with a cooperative device".
Right now there are huge market incentives to make devices ever more uncooperative. Unless something changes, the choice for consumers will likely be soon "tolerate an uncooperative device and don't block it - or opt out of technological progress altogether".
> That's fundamentally not possible; any mechanism usable by 'good' network admins can be used by 'bad' network admins.
This argument doesn't get any more coherent no matter how often it's repeated. Somehow the tech industry keeps arguing that the key to any kind of mandatory backdoor will leak with mathematical certainty - while employing that exact same kind of backdoor for their own purposes without any worry - in the form of forced updates and forced telemetry. How exactly can the same technology be a security risk when used by a government but a security best practice when used by a private company?
> We absolutely need full control over our own devices, but many of the people most capable of fighting for that control got complacent about it because a subset of people could hack around it with local network interception stunts.
So then what would be your idea how to gain control?
This isn't new to DoH. Malware has used alternatives to DNS for getting command & control server IPs for decades. IRC used to be common, for example.
IRC easilly slotted and controlled.
Yes. Which is why it's no longer as popular. The point is that malicious actors have never followed the system's preferences on how to look up server IPs, so why the hell should they start now? DNS blocking hasn't worked up to now, why complain that it's suddenly broken?
The answer is - it depends.
If you own the device(s) (your internet connection, phone or a colo), and it is someone else's network - you want security.
If it's a device (smart tv, media player, iot) that you don't "own" on your network you want control.
The 'historical quirk' per my understanding is pretty much IPv4 NAT.
At least, I couldn't figure out how to do it with IPv6 (no (need for) NAT) - I ended up dropping them if not destined for my desired DNS instead.
(NAT lets you Translate Addresses, usually to save IPv4 space, but here to redirect to a different DNS. IPv6 fixes the address space problem with more addresses, so the hack is done away with, and everything on the network can 'route itself' to everything else without any translation, as pre-NAT and as always intended.)
Transparent proxying works for IPv6 as well as IPv4. It's a little challenging to do transparent proxying on a box that's not the firewall. If the firewall and the proxy are on the same ethernet broadcast domain, you can forward the packets (i think for Linux you'd need to do policy routing on the port?, for ipfw there's a forward action, i think pf would be route-to); and if they're not, you'd need to tunnel the packets to the proxy.
NAT doesn't really have much to do with it, other than you with NAT, you certainly have a device on your network that's capable of being a transparent proxy, whereas without NAT, you could just have a unmanaged switch to share your upstream connection between multiple computers; assuming your provider is enlightened and provides ethernet or an ethernet bridge with nothing else in the way.
> think for Linux you'd need to do policy routing on the port?
I think that's probably the crux of what I'm getting at - and I should say this is all from fiddling at home and not working in networking - with iptables you can `-j DNAT --destination $desired_dns_ip`, but if you don't have NAT then you're left with `-j REDIRECT --dport $portchange`, which yes, takes a destination port, not IP.
> other than you with NAT, you certainly have a device on your network that's capable of being a transparent proxy
Yes exactly, that's all I meant really. After all, the context is a home user configuring a home user's 'router' (i.e. switch, firewall, AP, ...) to send DNS requests to PiHole, right.
The first part is really just a tooling issue. iptables doesn't have a -j FORWARD to pass the packet as-is to another host, but it's certainly possible to make it happen. I didn't look at nftables to see if it does either.
Yeah absolutely, as I said in a another branch of this thread, it's all just (plaintext even) 1s & 0s at the end of the day, you can flip some; I do understand that.
It's just that our starting point here was something like 'what can a software engineer do in an hour or so one evening to get their home DNS traffic going through a Raspberry Pi on the LAN'.
I happened to have, er, exactly this experience recently with my (OpenWRT) router(+...) and with that tooling, I couldn't make it work. I settled for DNAT for v4 & DROP^ for v6 because that's what I could get done that evening and it didn't (and hasn't) seem(ed) to be a problem.
^(as in, anything trying to use a different IP, if it's already destined for my DNS that's fine. The hope being that it falls back on v4 at least, even if not the provided DNS over v6.)
Any firewall worth its salt can do this with IPv6 as well. There's nothing on a technical level preventing it.
iptables?
Of course it's technically possible, it's all just packets of data that you can change, at the end of the day. I just meant that the easy way to do it (AIUI) went away with IPv6 because you were never supposed to need to do that, and with v6 you 'don't'.
It's not like the purpose of firewalls is redirection.
I thought that iptables was the easy way to do it? (Both for IPv4 and IPv6.)
NAT66 is an experimental (10 years old IIUC!) RFC for one-to-one IPv6 mapping. While there's no RFC specifying an official method of masquerading behind a single address, AFAIK iptables "just works" (as long as you aren't running FreeBSD).
Create a loopback interface and assign it the IP address you want redirected, then have your DNS resolver bind to those IP's.
Yeah it is true, but the problem is if DoH with its privacy features come along it will be literally impossible to block ads with PiHole, as all DNS quesries will be encrypted (which DoH does ofc rihgt now).
So no ability to block ads in TV or malicious domains that IoT deviced connect to or even blocking Windows, macOS telemetry network wide as all DNS requests will be encrypted which they are not as of right now.
It will be possible, just harder. I will not be surprised if there will be services where one can get a database of IP addresses of bad domains and one could block that, the same way one does with spammers. Surely it may block sites that share hosting on those domains, but it then it puts pressure on sites to avoid such sharing sharing.
> but it then it puts pressure on sites to avoid such sharing sharing.
Not really if that hoster is Cloudflare.
On every single operating system it is possible for this kind of improvement to be installed as a system-wide replacement for the local resolver, whether by a direct plugin or by running a resolver on localhost. This is how these upgrades can be deployed if you don't want to wait for the OS.
The problem is that browsers and other applications are just unwilling to let the user see how their products work or decide anything for themselves--or even just architect their installers to involve dependencies on a shared resolver upgrade--and so we end up in this hell of applications actively hiding their traffic from you.
And like, "great": now we have a new version of DoH and have to wait for everyone to upgrade their apps that upgraded to DoH before? This is ridiculous bullshit... this should be a single app on your device you now upgrade. Hell: Cloudflare even develops that app for a number of platforms! They aren't even the problem... it is everyone who jumps on "embedding" this behavior :/ :/ :/.
(For a more technically-comprehensive rant about this, read my comment from a year ago:)
iOS and MacOS technically have this (requires a profile[0]) but Microsoft will probably drag their feed on this for the next 2 years with the amount of enterprise customers they have to keep happy; and, given that the network adapter config is still based on Aero controls, they're probably in no rush to add more configuration options before upgrading it to Metro controls.
It's almost ready in Windows.
https://techcommunity.microsoft.com/t5/networking-blog/windo...
nice to see - looks like it's even been ported over to the settings app -
https://blogs.windows.com/windows-insider/2020/08/05/announc...
I still think Chrome and FF having it is a net positive though - The steam hardware survey shows 7% of users' OS is a Windows version before 10, so all of these users can use it today before it gets to them in the next major Windows 10 update.
? On Windows this is trivial to implement as you can just set the resolver to DNS on localhost; the only reason iOS needs something special is because you can't run servers/daemons.
This is exactly why browsers support doing their own name resolution: because some OSes advance much more slowly.
But instead of it being in the browser, it should be a separate app that upgrades all apps that use the native resolver.
Why on earth is a "configuration profile" required for this?
I imagine Apple is weary about having a UI for this since it might cause more bad PR/complaints about people being able to get around (for example) school website filters. Chrome eased its UI rollout by introducing a managed policy that admins could configure before the UI was enabled. https://cloud.google.com/docs/chrome-enterprise/policies/?po...
Because if you need to configure a fleet of Macs or iOS devices for a school or business, it’s easy to push a configuration profile to all of them using tools commonly in use by IT departments and admins.
> Querying nameservers is being pushed into applications themselves...
I agree: this absurd trend will lead to every app essentially including an entire O/S. There are several reason why OSes provide services to applications and one is that the OS manages the user's configuration (e.g. what devices are plugged in, where and how to resolve names, cacheing data, etc).
I also find it rather insane the amount of overhead required to resolve a name when an entire http connection needs to be set up and torn down for the process.
I presume there’s some kind of connection pooling going on ... but yeah you’ve still got the needless verbosity of HTTP for a specialised high volume protocol ...
When enterprises own the devices on their networks they can add whatever root certificate they like and MITM filter whatever they like. This makes over the network traffic more secure and does nothing to really impact threat detection.
If you don’t own the device then 1) should you be interfering or snooping the traffic at all? 2) if you need to limit threat then subnet clients you don’t own.
Adding in root certificates and performing HTTPS MITM even on your own network is a huge risk on its own. You're basically breaking many layers of security such as certificate pinning that just get disabled with any custom root certificate. Those middleboxes will always break client certificate authentication and you're trusting that they are checking the various revocation lists they should be (CRL/OSCP/various built-in blocklists).
For your basic premise to work, it also means that the MITM middleboxes will need to support the DoH protocols and support recording, and filtering those responses.
Additionally, custom roots certificates will only work on devices that you can actually set a custom root certificate for, a great example is IoT devices. Is your TV suddenly talking to a botnet or was that a legitimate update?
We can argue about whether those middleboxes are even sane to deploy (hint: they're not), but what is historically true is that they are known to update slowly to new protocols, if at all and are known to cause compatibility issues for traffic that is inherently expected to be unchanged. They're enough of a problem at the _TCP_ layer that QUIC was explicitly designed that minimal information is available in the protocol headers so middleboxes would have less to mess with.
oh don’t get me wrong, i’m not advocating for root certs and MITM boxes. i’m just saying DOH isn’t really going to challenge an enterprise’s ability to sniff and intercept DNS.
notwithstanding other technical issues, it’s just bad practice to create the sort of experience MITM creates.
when people are used to seeing compromised https when on the corp network or mitm boxes prompting for auth periodically it basically lowers people’s guard on that stuff and opens the door.
Not with TLS1.3 and DoH and ENSI they can’t.
ESNI is a draft. No support in Chrome and OpenSSL.
do you own a smart tv in your house?
The problem is that the things you most want to block can trivially bypass your local DNS filtering - DoH is just standardizing something which has been done for decades.
The only effective measure is to block outbound network access and require use of a proxy, possibly optimized by allowing direct traffic only from clients with functioning endpoint monitoring agents.
DoH is highlighting the security nightmare that is AWS, GCP, Azure, Cloudflare etc with their reverse proxies and virtual hosts, making it impossible to safely restrict a network from communicating with only a specific cloud-hosted service.
Also often perfect security isn’t required. Doors with locks are good, but useless if the burglar just breaks the full length glass window next to it. They still serve a purpose, but don’t need to be an absolute comprehensive solution.
The biggest mistake I see in public cloud fundamental architecture is that it uses IPv4 end-to-end, instead of just as a compatibility add-on to native IPv6 networking.
Literally all of the security issues caused by the public cloud network architecture instantly evaporate with IPv6, as well as much of the configuration complexity.
No more private networks with non-routable addresses! Instead you get a public-routable IPv6 block.
No more split-routing issues.
No more "gateways" or "peerings" or "service endpoints".
No more Private DNS Zones that may or may not work across virtual network boundaries.
No more copying DNS records into on-premises Active Directory DNS.
Every VM can get a globally unique address. So can every service, of any type! No more conflicts. No need to carefully "carve up" and "allocate" addresses. Just let the system take care of it...
No more sharing IPs with other customers. Every resource, no matter how tiny can get a dedicated address. Got an S3 bucket with 1KB of files in it? You get your own IP!
Every VM or service sees the real client IP, not the reverse proxy IP.
No need for SNI, ESNI, or even host headers since every web server can have a dedicated IP.
No reverse proxy means that load-balancers can simply set up the TCP handshake and then everything runs directly at wire speed. There is never a need to "scale" a load balancer.
The IPv6 addresses are consistent, globally. The IP address of the cloud VM is the address you register on-premises to SSH to it. No NAT magic involved at any point.
Adding a private link (e.g.: ExpressRoute) doesn't change your address ranges. They're the same, only the routes change. This would be a completely transparent change to your firewall rules of whitelisting setup.
Etc...
PS: The current Azure IPv6 architecture reproduces all of the limitations of their IPv4 architecture. They even NAT the addresses! You literally cannot have any of the above, ever, with Azure using IPv6 as it is now. They even limit the number of IPv6 addresses to further restrict you. If they do fix it, you'll have to redo your entire IPv6 setup. It's insanity.
Thanks for spelling it out like that - I think I might see the bigger picture here now. DoH (which incorporates DNSSEC under the hood) to protect the name lookup. IPv6 to provide unique addresses for every single service you connect to. The complete removal of SNI (as a security threat) and ESNI (as unnecessary complexity).
Pi-hole type filtering is then implemented based on IP blocks instead of DNS queries. Any unrecognizable IP address is default denied. Tracking, analytics, and ads could still be proxied by a remote host, but that can already happen anyway.
Of course, your ISP (or VPN, or anyone else along the network path) could employ the exact same approach to determine the services you connect to. Which leads me right back to DoH being largely pointless and Tor or similar being a hard requirement for actual privacy. Unless I'm missing something?
Due to consumer privacy concerns there will always be people wanting a high degree of anonymity when accessing services, regardless of IPv4 or IPv6. Both from consumers and the people running the services. Right now CDNs provide this privacy using TLS with ESNI. This won’t go away with IPv6. Likewise with DDoS concerns some will want to put CDNs and cloud reverse proxies in front of their services, intermingled with lots of other services.
The solution of dedicated IPv4/IPv6 is necessary for proper network control. But for the reasons above many won’t do so.
Also IPv6 is fundamentally not ready for real world use within small/medium businesses and homes IMO. I know you were talking about IP networking within clouds, forgive me, this rant isn’t aimed at that, it’s just my general IPv6 is not ready rent. At least not without NAT. Why?
- Can’t just simply put one IPv6 router/firewall behind another. Not all IPv6 routers support DHCP-PD, and even if they did, you could have 2-3-4 levels of routers/firewalls at a business. I’m not making this up- retail/gas/food industries often have a plethora of networks at a location, and the business/franchise owner is not tech literate, or even if they were, the equipment is managed by third party vendors and they don’t want to customise their IP network for each location. It makes for messy deployment and maintenance.
- Can’t just simply open a firewall rule on the main site router to forward say HTTPS to an internal service. Why? Not all ISPs give static IPv6 prefixes, not all PCs/servers/devices support DHCP6 for static leases, and then there’s IPv6 privacy addresses. Yes, you can statically configure (only if ISP is static too!). No, I don’t want to open the port to all devices on the LAN and no I can’t rely on each device to be running their own firewall (let alone a properly configured one!).
- WAN failover / multiple ISPs is hard. You have a fibre primary feed, and a secondary cellular/5G feed. Each has different IPv6 addresses. How do you ensure the right ISP is used at any given point? IPv6 shifts this decision to the client. This makes load balancing and policy based traffic routing (eg VoIP over fibre 1, FTP over fibre 2, etc). Also the cost of using a multi-homed IPv6 subnet & BGP in a SME/retail business is out of the question (plus the cellular ISP wouldn’t support it anyway).
All the above works fine out of the box with IPv4 and NAT. It’s bread and butter easy. At the cost of not having dedicated unique public IPs but these places simply don’t need them.
What IPv6 should have done to ensure a smooth migration is allowed for NAT from the very start. That would have let everyone who needed public IPs get them straight away, and those that didn’t to still migrate anyway with as little drama as possible. But it’s just not the case, NAT has been slowly added but it’s support is far from ubiquitous that IPv4 has.
> Due to consumer privacy concerns there will always be people wanting a high degree of anonymity
"No one can have glass windows in their homes, because a few people like to walk around naked at home, and they're worried about their privacy."
or
"You can't have steak, because if a baby were to eat it, they might choke."
> Right now CDNs provide this privacy
Nothing at all stops you using CDNs with IPv6.
> Also IPv6 is fundamentally not ready for real world use within small/medium businesses and homes IMO.
The protocol has been ready for 10+ years. I'm on an IPv6-enabled home network right now, and it works just fine.
As for the rest of your arguments: They're a side-effect of there being insufficient pressure on ISPs to do their jobs.
If public cloud providers primarily used IPv6, that would very rapidly force ISPs to get their act together and fix their woeful IPv6 support!
> DoH is highlighting the security nightmare that is AWS, GCP, Azure, Cloudflare etc with their reverse proxies and virtual hosts, making it impossible to safely restrict a network from communicating with only a specific cloud-hosted service.
This problem goes back a lot further than that — even prior to having CDNs in common usage you had plenty of different clients sharing IP space at hosting companies and the really malicious stuff just using compromised computers. It's also not fully covering the threat model here: for example, if you are concerned about privacy there are whole classes of device which you simply cannot allow because blocking one feature simply means that the vendor will run everything through the same endpoints required for the device to work.
> Also often perfect security isn’t required. Doors with locks are good, but useless if the burglar just breaks the full length glass window next to it. They still serve a purpose, but don’t need to be an absolute comprehensive solution.
Perfect is the enemy of the good but you have to also think about the asymmetry here. Trying to hijack DNS is only useful when you control the network but neither the client or server. Trying to stop malware by politely asking them to play nicely is a losing game and the IoT devices many people worry about have entire teams devoted to bypassing you as well (e.g. if any significant number of people tried to block a TV from reporting your viewing activity, the endgame is that viewing history and software updates would both go through samsung.com, not that they'll give up millions of dollars in revenue). That leaves you with cases where you do control the client and thus have less invasive alternatives such as installing an ad blocker or using a different browser.
> Trying to hijack DNS is only useful when you control the network but neither the client or server.
This is a very common scenario in retail and SME businesses where you can’t control the on site hardware, eg fuel pumps or CCTV or cash registers or footfall counters or vending machines or whatever, and you can’t control what IPs they talk to. But you still have an obligation to minimise their network access.
Unfortunately they use services hosted on the cloud in dynamic IPs, so you either need to MITM TLS (can’t, no way to change the trusted CA list in these devices), or you need to MITM DNS / SNI. It’s not perfect security, and ideally the business simply wouldn’t buy such poorly securable products, but technology decisions are very rarely vetoed by security considerations, especially when cost is a factor. This is an example of the appropriate level of security for the business risk appetite. Telling the business to buy products that can be locked down perfectly is normally a non-starter.
Again, though, this is only forcing them to understand how their network has always worked rather than changing the situation. If the vendor chooses to bypass the local DNS, you won’t know about it no matter whether DoH exists.
How can they trivially bypass this local filtering? If the router is redirecting all port 53 traffic, there is no way to bypass aside from some alternate name resolution scheme.
If the network allows outbound traffic, they can hard-code an IP list - this is how Cloudflare’s 1.1.1.1 works and malware has done this for decades – or they can use local DNS to resolve a single name which will answer or redirect to a service which does further queries. Malware commonly used IRC for this until that started getting blocked on most networks, but imagine how easy it would be to miss, say, a bot which connects on 443 to a major hosting provider, like half the apps you run, searches Google.com, Twitter, etc., or hits an ad network for a keyword selected by the attacker.
In every case, once they get the server(s) to connect to you lose all further visibility unless you’re blocking 443 and forcing traffic through an inspection proxy.
Yeah, it’s an arms race, but I suspect it’s solvable: at least solvable enough that it’s feasible to just not use devices that break your policies. For things like Pi-Hole, the setup I describe will reduce much of the ad noise even without more complicated systems.
The way to solve is either to segregate unmanaged devices onto a separate network and give up on controlling them or to implement the system I described. The same Pi running a DNS server can run a proxy which applies blocking policies on all hostnames.
> The same Pi running a DNS server can run a proxy which applies blocking policies on all hostnames.
With DoH, it's easy to not just hardcode the resolver's IP but also the resolver's certificate. How would a proxy be able to intercept that?
If you configure an HTTPS proxy, the client will use the proxy's name to verify the connection to the proxy and trust that the proxy will verify the remote connection.
If you're trying to configure transparent proxying where the network redirects traffic to a different device, you would need to have a local CA so you can forge certificates — that's not uncommon in enterprise IT but it's definitely a security risk associated to having something which can MITM anything on your network.
In either case, the real question is whether you control the endpoint. If it doesn't support configuring a proxy or installing a CA, all you have is the binary decision to decide whether or not to allow it on the network at all since whoever does control the client has so many options for smuggling traffic out.
> If the network allows outbound traffic, they can hard-code an IP list - this is how Cloudflare’s 1.1.1.1 works and malware has done this for decades – or they can use local DNS to resolve a single name
Why can't you redirect all 53 traffic to a pihole and block that single name?
In this case the client isn't sending traffic on port 53 at all — they already have the IP for the target server and so they just open a connection to it on port 443.
This is not hypothetical: it's how DoH works now but it's also how various things have worked for decades. Malware liked it for hiding command-and-control name queries from the few people who monitor DNS but it was also an option for anyone who had problems with buggy or malicious local DNS servers to add public resolvers like 8.8.8.8 or their own infrastructure into the search list so they didn't get support calls due to some ISP breaking their own DNS server.
The key part is remembering that this was always possible. DoH just meant that more people became aware of the gap they'd always had in their network management.
Based on this: https://stackoverflow.com/questions/33099569/how-does-sock-5...
> Firefox, for example, sends hostname to SOCKS proxy without resolving it.
So a simple proxy can bypass local restrictions.
-----
I remember doing something similar back in my IT days just to see if it was possible. It was.
> If the router is redirecting all port 53 traffic
Then it won't do anything to DNS over HTTPS traffic that is going over port 443. And it won't be able to distinguish that traffic from any other HTTPS traffic.
There’s theoretically ways to solve that: https://news.ycombinator.com/item?id=25349358
How well does the redirect scheme work for a device that connects to a central DNS server listening on, say, port 5353 instead? What about 80 or 443?
Well, it’s more complicated, but in theory you could do some deep packet inspection that understands the protocols: personally, I’d use this to break DoH connections (for every host name seen in SNI, attempt a DoH query, if it resolves, reset the connection) and attempt to force everything to fall back to plain DNS. Then, whitelist a couple outbound ports (on most networks, maybe just 443 + 53?) and block VPNs.
With the Portmaster (https://github.com/safing/portmaster) we're going in that direction, but it will take a couple more years to be able to go that deep. Have a look!
> or every host name seen in SNI
Not going to be possible in a few years or so:
meh. The outer SNI and the IP address still tell a lot about what you are doing online.
Also, with things like this, you can just reset connections using HTTPS features you don’t support. It might eventually become painful, but it’ll be fine for the near future. And, if enough enterprise middleboxes do this, the standards will be DOA.
> How well does the redirect scheme work for a device that connects to a central DNS server listening on, say, port 5353 instead? How about 80 or 443?
Just because it is not a silver bullet doesn't mean it is not effective for a large percentage of users.
Pi-hole is still very much niche, and outside of that and enterprise network admins, the majority of users are stuck using the trash resolver of their operating system directed to the trash DNS servers of their ISP.
If DNS level blocking ever becomes popular enough, malware authors will change their systems to not use the system's DNS, or to use hard coded DNS over TLS/HTTP servers that they know will serve them the data they want.
Preventing proprietary applications from doing malicious things is a constant cat and mouse game. Pi-hole works well for now, I would argue, because it's not popular enough to put a big enough dent in the success rate of malware.
DNS level blocking is extraordinarily common in enterprise settings though so the cat and mouse game for malware trying to avoid it is decades in the making at this point, pi-hole is just a recent common name in the prosumer space.
pi-hole is the most recognized name on here, but I think you're wrong about DNS level blocking. There are lot of home routers that have started integrating that feature natively. A few examples that I know off hand:
- Motorola MH7022
- Eero
- Disney Circle
Basically any of them that say "content filtering" in their product sheet are using DNS level filtering.
> ...the trash resolver of their operating system...
I'm very curious why you think the well tested, vetted, and constantly updated resolvers built into OS's are "trash". Is it just because most don't have support for DoH or DoT?
Most of the OS level resolvers just forward to the resolver provided via DHCP. Which is fine for enterprise, but for home users means using the ISPs resolver, which can track users and has a history of injecting ads on failed resolutions.
Not to mention the user interface for configuring them has never evolved beyond /etc/hosts and the total disaster that is /etc/resolv.conf (on modern linux distros).
The fact that I have to run a separate device such as pi-hole to intercept DNS rather than just point my OS resolver to a blacklist indicates how OS level resolvers have not kept up with the use cases people are asking of them.
And of course, no support for DoH or DoT or even DNSSEC.
It's a good thing that network-level filtering is getting harder. If you can use network-level filtering to block ads/trackers/malware from your own devices, countries and ISPs can use network-level filtering to censor other people's devices. If you want to protect your own devices, do it locally, even if it means configuring Firefox instead of just the OS.
I don't know much about the tech, but would it be possible to setup your own local DNS server that your machines point to, and do the filtering within that DNS server?
I run my own recursive resolver (DNS server) in a datacenter.
This DNS server gets its upstream resolution from nextdns.io which is configured with several blocklists - including one that is roughly analogous to ublock origin.
On my local network, my DHCP server hands out my DNS server to all clients.
This means that all clients on my network get fairly robust ad-blocking even if they do not have an adblocker installed. It also means that non-browser clients (Sonos, AppleTV, etc.) get ad/tracker blocking as well.
DoH sort of breaks all of this, unfortunately.
Individual devices or clients or browsers can now connect to trackers and ad-servers over HTTPS, bypassing my adblocking resolver.
I thought that perhaps there was a solution wherein you would pre-query every single new IP you connected to over HTTPS and send it a test DNS query .. and if it answered DNS, you would just refuse to talk to it. I think this falls apart, however, if (for instance) google just queries "google.com" ... now you're denying google.com because it answers DNS queries over HTTPS...
Look, back when 8.8.8.8 came online I could just smell it ... I knew there was a user-hostile arms race somewhere in there I just didn't know where. Now we know.
Google public DNS (8.8.8.8, 8.8.4.4) was created because OpenDNS was redirecting "Google queries" typed into the browser GUI (rather than typed into an HTML form on a website).
It is not difficult to run own DOH server. For example, dnsdist >1.4 seems to be a popular choice:
https://blog.apnic.net/2020/02/28/how-to-deploy-dot-and-doh-...
Yep! If you have a Raspberry Pi, look into Pihole. Some routers have this capability too.
> bypasses any system resolvers configured on the host
With the Portmaster (https://github.com/safing/portmaster) we actually tackle this problem by notifying software (eg. Firefox) or blocking their connections, forcing them back to plain DNS, which we can redirect and handle. Take a look!
Applications still have fallback though, right?
If so, I foresee blocks on DoH/etc to common resolvers like 8.8.8.8 and 1.1.1.1. I'll be blocking them at home on the assumption that I only want regular DNS lookups so I can point them to my own DNS server etc.
Firefox and Chrome do yes, but none of these standards specify any kind of fall-back behaviour. There are no guarantees that any specific device or application needs to do that or how.
There is a cost to firewall rules like that as well. Who is going to maintain a list of all the IPs on the internet that are hosting DoH servers that could be used? What about the potentially more prolific proxies specified in this protocol enhancement? How does a network administrator keep all of those in sync with their edge networks? How does a home user?
Since DoH uses HTTPS there is no reason a service can't be multihomed on the same IP just like SNI allows multiple HTTPS servers on one IP. Would you be willing to block a legitimate website just so the applications on your network might fall-back to the name server you want them too?
You can snoop SNI and use that to block traffic, right? In another thread, someone suggested also stripping the keys for encrypted SNI from DNS queries which, when combined with a firewall that attempts to make a DoH query to every new SNI name it sees might potentially work.
For now. Encrypted SNI is in development and will eventually be used by devices and apps that want to do this sort of thing and hide what they're doing.
The solution to that is pretty simple. ESNI, I believe, requires encryption keys in DNS records: if you control DNS, you control ESNI. ECH might be harder to deal with, but you can always just block HTTPS connections you don’t want to support. Also, will some sort of certificate fingerprinting still work?
Well, adblock lists exist, DNS block lists exist, this is just something along similar lines that can be tested and aggregated and distributed. Not a perfect solution, but pi-hole as a router or similar could do it.
"If so, I foresee blocks on DoH/etc to common resolvers like 8.8.8.8 and 1.1.1.1. I'll be blocking them at home on the assumption that I only want regular DNS lookups so I can point them to my own DNS server etc."
I wish it were that easy but it's very predictable that google will start resolving DoH on plain old "google.com".
So will everyone else ... it's not going to be malicious.userhostile.resolver.samsung.com, it's just going to be samsung.com ... so you can block it but you'll be blocking more than just the resolver ...
Grim thought, but surely they'd have to be serving from the same IPs?
Right, but do you want to block all access to google.com ?
I, and many users on my home network, use that site a lot :)
Blocking traffic for known DoH services would be trivial. How about blocking unknown, how would you block that?
Think out of the box: Just don't let the app connect to an IP it has not resolved a domain name for.
That's what we can do with the Portmaster (https://github.com/safing/portmaster). Check it out!
Thanks, I will check it out.
Trying to do content filtering on the kids chrome books for remote learning, I installed a pi-hole and was generally pleased with it. Then the kids (8 & 12) figured out to change the dns to cloudflare or google directly. I ended up having to add static routes on the router to block those paths.
(I need to block distraction sites during instruction time - otherwise it’s endless Minecraft videos while in zoom meetings...)
Any "how to" recommendations for setting up something similar? I have a pi-hole but after reading this thread I feel like im severely under-utilizing it.
I have blocked 1.1.1.1 and 8.8.8.8 and noticed some devices behave very badly, often crashing or restarting. Debugging the issue, once I removed the firewall rule they behaved normally. Almost all of the affected devices were google related, Android TV for example.
You can also DNAT all port 53 traffic to your resolver. Devices will think they are talking to 8.8.8.8 or whatever, but in reality they will ask your resolver and your filtering will apply.
Your filtering can still break these devices.
This only works as long as DNS is both unauthenticated and unencrypted.
Which, at port 53, it is.
Obviously, it would work with DoT (853) only with cert verification disabled.
Enterprise can disable DoH in Chrome using a group policy.
Enterprises can also disable direct outbound connections to anywhere. They can insist only way to access anything is via http proxies. You can do the same at home with a pi-hole. On iOS/MacOS you can install content/network filters. If a browser or app breaks these basic functionalities of Internet, then good luck to them.
You're right, that doesn't help with Apple devices though which has seen a huge market share increase with day to day users in the businesses that I interact with. Same thing goes for mobile devices, even those with enterprise management systems attached.
The options are simply not there for consistent management of your network.
Can't you do something analogous to editing your hosts file (but for name servers) and then re-centralize all of your apps using a single name server, which you can then control however you like?
No, because instead of setting your network to give out your DNs server to all of your devices, it’s ignored by the apps
Instead of setting your devices to use a dns server of your choice it’s ignored by the apps
Some apps allow you to configure them, so now you’re configuring 200 apps on 20 devices rather than just one dhcp setting.
(Oh and OSs have generally broken hosts files)
Sorry, what I was trying to say is: you know each app tries to use a certain DNS server. So, in your Rasp Pi, you route their DNS server to point your own (as you would with an /etc/hosts file), that way when DoH occurs, you control the final resolution.
What I'm suggesting isn't merely setting up the 'default' dns server. What I'm suggesting is 'cnaming' the name servers that apps attempt connecting to, to point elsewhere.
If you know the iP the DOH client uses you can intercept it. But you can’t spoof it without breaking TLS, which means deploying your own certificate
This general industry move will lead to more tls breaking proxies and more network interception. All because people don’t want to understand how a network and os work.
Making doh at the application later normal it moves the power towards the centralised advertising network and away from the individual. That the SV culture likes this is unsurprising.
Giving the user the ability to expose these requests by sending an HMAC of the domain to a local resolver could help with the privacy portion, but still allow for custom routing based on context.
why should DNS be handled at the system layer and not by applications? There's zero controls in place to stop this so I don't see why it's assumed that every application developer will want to use system defaults and not override it.
> I don't see why it's assumed that every application developer will want to use system defaults
It's the user's machine not the application developer's.
And users get to decide which applications to install. Tunneling has been a thing for decades; likewise for malicious programs. Vigilance when installing programs on a networked device has always been and remains necessary.
But it’s not under the users control if they install an app- there’s nothing hard that prevents the abuse. Now if the OS had a system wide / network level proxy that checks the correct DNS calls are getting made and overrides with a user chosen default, then you’d have something.
But we don’t, we just have a default
Because, DNS, like default gateway and ip address, are configured by DHCP? Zoned DNS server for intranet is very common.
It bothers me how "privacy" has been redefined in recent years to mean "encrypted" and not "surveillance-resistant". We keep building things that make more requests I can't terminate locally, e.g. to a PiHole.
Never forget the lesson in "Using Metadata to find Paul Revere": https://kieranhealy.org/blog/archives/2013/06/09/using-metad...
As another HN user put it: https://news.ycombinator.com/item?id=25349426
> Administering devices with network settings is convenient, but rapidly vanishing because there's no technical difference between you administering your local network and a totalitarian ISP administering their users.
Your ability to terminate things locally means that finding Paul Revere with metadata isn't needed. It's a lot of work when you can just directly look at all your country's traffic.
Until the country where all those network connections terminate turns totalitarian...
Or until your country makes some friends and they decide to share their piece with each other until they recover your whole request.
There is no system that both allows communication between multiple parties and is infinitely resilient to a sufficiently determined, large, and well-funded group of people with guns.
What we can do with design and architecture is make each breach of communications security consume greater resources than unencrypted DNS, unencrypted HTTP, and unencrypted email.
There is no bit of land left on Earth where I could go live and not be governed by a determined, large, and well-funded group of people with guns. Perhaps it's time to start building with them in mind first?
Quite. That said, migrating to new protocols and systems is rather challenging. So we get retrofits like DNS over HTTP, and Oblivious DNS over HTTP.
Historically, radicalism in the form of whole-protocol-suite replacements has often seen little in the way of adoption. Whereas incrementalism in the form of tunnels and other protocol amendments has been much more successful.
There actually is unclaimed land left on Earth. I know there's a bit between Egypt and Sudan. Claiming the land would worsen their claim on a different, better, piece of land that they both claim is theres; so neither does.
Unsure of the status of non-government large, well funded groups of people with guns there is though.
People in totalitarian regimes aren't safe. This is kind of a given.
What everyone everywhere can resist, though, is corporate surveillance. That's the aim people should have.
You're absolutely right. Corporate surveillance can, should, and must be resisted.
It is perhaps worth considering carefully if exposing most people to surveillance they are not equipped to defend against from lots of entities (such as with typical DNS) is an improvement over something like Oblivious DoH. It might even offer some protection against some totalitarian governments in some cases.
Again, you're completely right. Protecting people from corporate surveillance is very important! I just think it might be worth considering that we don't have a perfect answer at the moment, and some subtlety in how we think about this might be in order.
Yes. The alternative is what I call "totalitarian liberalism" - it's not going to be communism, it's not going to be fascism, it's going to look a lot more like IKEA, and while it may be easier to physically step out of line, the incentives structure is rapidly converging on finding ways to keep you corralled in invisible ways.
I've not read that analysis before, thanks for the great read!
This is a neat design, but, does this not just shift the issue of trust as to whether the proxy and the target are colluding:
> However, each of these guarantees relies on one fundamental property — that the proxy and the target servers do not collude. So long as there is no collusion, an attacker succeeds only if both the proxy and target are compromised.
I'm not sure how an end user would be expected to assess this any more than they could ascertain whether any particular DoH/DoT provider is as trustworthy as they claim.
Exactly what I was thinking. It doesn't even really help to run your own proxy on a server somewhere, because although the target wouldn't know for sure what the client's IP address is, queries from just one IP are likely to be easily correlated (statistically or otherwise).
So you convince some neighbors to use your proxy... As the number of clients grows, so does the uncertainty that the person running the proxy isn't colluding with the target, so you're back to the same trust issue that you were trying to solve in the first place.
Add a few more proxy hops and you’ve effectively reinvented Tor
.. but the version of Tor which does not raise alarms on every corporate firewall/IDS system
reinvented Tor, but with extra steps and less security
But probably faster.
Yes, just like a proxy or a VPN.
Faster - but it does not solve the problem.
Well, this could probably encourage the creation of privacy-oriented proxys (they just have to forward queries, so it should be relatively inexpensive compared to a full DNS server). What is the likehood of someone getting logs from Cloudflare (who promises it does not keep logs, but let's assume it does) and at the same time hacks into some random privay-oriented organization?
Of course, one might imagine a State actor using all their resources to do just that. But this would be a very complex attack. At least, it would stop all kind of ad tracking.
The worst part of this proposal is that it will further centralize the DNS infrastructure.
What a stark difference between Google and Apple/Cloudflare.
Apple/Cloudflare are working on privacy-friendly protocols that reduce the amount of information exposed to them.
At exactly the same time, Google is working on proxying browser traffic through them without any consents [1].
I guess all the user "consent" for Google proxying (spying) users are included in the ocean of ToC text
Thus I am using Google Chrome only for Web Dev
Opened this post expecting to be hating on another power grab dressed up as protocol engineering, but this one seems to actively /reduce/ the centralization of user data collection in DoH. Props to Cloudflare, I'm impressed.
All I see is a proxy service and a way for cloudflare to get access to the data
The proxy sees the client IP, but can't look at the encrypted DNS request.
The DNS server sees (deciphers) the DNS query, but not the client IP address.
It's a proxy, but with the sensible data encrypted with the server's public keys to hide it from the proxy. Cloudflare never knows who is sending the requests. How can they get access to the data?
While individual clients may not be easily identifiable, there's still a measure of identification that could be made, if you were to configure the public key DNS server to send a different (but persistent) public key to each IP address which asks for the DNS record. (Probably an ISP's caching nameserver.)
You can't tell how many people are going to be covered by that public key, but you could probably make educated guesses, or combine this with other metadata.
They run both, or buy data from the company that runs the other half?
I'm not sure I see the point,tbh. If you want to control dns, why not resolve yourself, with whatever cache you need? And if you trust a company to do that for you - assuming the two companies do log "their half" - you're just a data breach, data broker agreement or an acquisition away from a commercial entity having all the data (again)?
Do you want Google and your ISPs to see everything? Cloudflare and maybe Apple (not sure what infrastructure they’d have in this if any)? Another company like Cloudflare?
I don’t know the answer but I’m curious to hear everyone’s thoughts. Personally I’d like to prevent Google and my ISPs but Cloudflare could easily become Google in many ways.
I am guessing most if not All future Apple devices / OS will default to use Apple Proxy for DNS?
I would like someone to correct me if I am wrong, but I think we can never have 100% privacy because the destination IPs cannot be encrypted or hidden, so as long as the destination IP can be observed, the server that you are connecting at can be obtained (I know a server can host many web pages, but this requires the port, which cannot be encrypted either).
So I don't know to what extent this protocol can be useful.
This is "fixed" in DoH the same way it's "fixed" for encrypted SNI: by having a small number of superproviders servicing millions of domains.
With current encrypted SNI proposal, your privacy (between you and the superprovider) is /improved/ by talking to a site behind a large aggregating provider. It sucks (since the superprovider still sees everything), but that's how it is.
edit: added clarifications in (parens)
I'm more worried about persistent, authenticated/ID-linked TCP connections (e.g. APNS) providing the client IP over time to an application service provider (e.g. Apple, Slack, Google, Microsoft, et c), that is, city-level geolocation track history via geoip, than I am the ISP or carrier snooping on what websites I connect to.
Every iPhone connects to APNS for push notifications and stays connected, and, last I looked at the protocol, the client certificate was linked to the device serial number. That's quite a geoip tracklog dataset, and AFAIK you can't turn it off.
It's to the point now that to keep my city-level location private from Apple, I'm not putting SIMs in any of my iPhones/iPads any longer, and carrying a battery powered VPN travel router (with a SIM uplink in it) for them to talk to. Super annoying that it has to come to this.
> but I think we can never have 100% privacy because the destination IPs cannot be encrypted or hidden
This problem is solved in I2P (https://geti2p.net) by adding a few intermediate hops between you and destination. You will know someone is connecting to the network, but you can't find what they're doing.
> I know a server can host many web pages, but this requires the port, which cannot be encrypted either
You can host multiple web sites in the same port since the 1990s, using name-based virtual hosts (https://en.wikipedia.org/wiki/Virtual_hosting#Name-based). It's rare nowadays to use a port other than 80 (for http://) or 443 (for https://) for public web sites.
yes you CAN if you wish of course. on the other hand nowadays 95% of domains on the Internet can be identified by IP[1][2]. so ISPs still have a pretty good guess what sites do you visit even without DNS or SNI data.
[1] https://blog.powerdns.com/2019/09/25/centralised-doh-is-bad-...
[2] https://blog.apnic.net/2019/08/23/what-can-you-learn-from-an...
I think there's still pretty good worth in this protocol. DNS is one of the key areas where we voluntarily give away information on every single website we're connecting to to a third party. This protocol certainly helps that--as long as the proxy and recursive resolver do not collude, neither will be able to associate the websites you're looking up with your IP.
It does have its limitations; a MITM can still just as easily see which IP addresses you connect to and determine which websites are associated with those IPs. But ODoH isn't really meant to fix that. A VPN would be better suited to fix that particular privacy concern.
The only solution is onion routing AKA Tor and similar.
"""A key component of ODoH working properly is ensuring that the proxy and the DNS resolver never “collude,” in that the two are never controlled by the same entity, otherwise the “separation of knowledge is broken"""
Essentially this is no better than using an HTTP proxy or a VPN.
A HTTP proxy (or VPN) know exactly who you connect to, even with SSL they know the target name since SNI isn't encrypted.
In this proposal the DNS-proxy doesn't know what you've sent to the DNS resolver.
I still have doubts, 1.1.1.1 was a clear power grab and effort to control more of the internet. DoH in partnership with Mozilla was an extension of that
So I am still suspect of their motives but maybe the negative PR got to be too much
Key bits from the Cloudflare blog https://blog.cloudflare.com/oblivious-dns/
> The target [resolver] sees only the [DNS] query and the proxy’s IP address. The proxy has no visibility into the DNS messages, with no ability to identify, read, or modify either the query being sent by the client or the answer being returned by the target. Only the intended target [resolver] can read the content of the [DNS] query and produce a [DNS] response.
> The whole process begins with clients that encrypt their query for the target using HPKE. Clients obtain the target’s public key via DNS, where it is bundled into a [SVCB/HTTPS] HTTPS resource record and protected by DNSSEC.
> Clients transmit these encrypted queries to a proxy over an HTTPS connection. Upon receipt, the proxy forwards the query to the designated target. The target then decrypts the query, produces a response by sending the query to a recursive resolver such as 1.1.1.1, and then encrypts the response to the client. The encrypted query from the client contains encapsulated keying material from which targets derive the response encryption symmetric key.
> ...50% of the time ODoH queries are resolved in fewer than 228ms.
BTW, DNSCrypt supports "oblivious" encrypted DNS queries via what it calls Anonymized Relays https://github.com/DNSCrypt/dnscrypt-proxy/wiki/Anonymized-D...
DNSCrypt needs meaningful industry support otherwise it's sadly irrelevant. I think by now we can all agree "industry support" basically means the 3 browser vendors. DoH has at least Mozilla and Google on board, and presumably Microsoft are tailing along.
If they allow to configure DoH server, you can use https://github.com/DNSCrypt/dnscrypt-proxy
> DoH has at least Mozilla and Google on board, and presumably Microsoft are tailing along.
Note that DoH (and DoT) shipped in iOS 14 and Big Sur, though aren't particularly easy to enable.
>Note that DoH (and DoT) shipped in iOS 14 and Big Sur, though aren't particularly easy to enable.
Specifically, you must install a properly configured .mobileprofile with HTTPS/TLS in the DNSSettings > DNSProtocol part of the payload (along with DNS server addresses of course). Merely pointing at a DoH/DoT supporting DNS server in the settings GUI won't do it, the OS doesn't do any probing and automatically use it just because it's available. For applications DNS Settings is covered under the Network Extension framework [0].
It's definitely nice Apple now has this built-in, and since they're onboard with Cloudflare/Fastly maybe this new twist will be pretty fast too. But obviously they're going to have to make this more automated for it to really make a widespread difference, ideally it'd simply see if the supplied DNS server (manual or DHCP) could run DoH/DoT and then just use it by default with no interaction required.
----
0: https://developer.apple.com/documentation/networkextension/d...
Also, macOS will not let you enable a DoH profile and Little Snitch (or probably any other tool using the Network Extension framework) at the same time. I don't know if this is a bug or intended behavior, but it's a disappointment.
Note that DoH (and DoT) shipped in iOS 14 and Big Sur, though aren't particularly easy to enable.
You can use something like iMazing Profile Editor [1] to create a .mobileprofile (which is just XML) to configure DoH or DoT.
Out of curiosity, what's the difference vs Apple's first party "Apple Configurator"? Do you like the GUI better, or does it expose more options?
I do like the UI/UX better; I've always found Apple Configurator to be clunky and non-intuitive.
Anyone have any idea why they chose to require 'configuration profiles' here?
Also, don't 'configuration profiles' require that your Mac have an associated AppleID?
Anyone have any idea why they chose to require 'configuration profiles' here?
There are several tools that can push configuration profiles to many macOS or iOS devices in one go [1]. It's also the kind of thing you don't want users in managed environments messing with if they don't know what they're doing.
Also, don't 'configuration profiles' require that your Mac have an associated AppleID?
I can't see why they'd be connected; being able to configure network settings isn't a "feature" related to having an Apple ID.
[1]: https://support.apple.com/guide/deployment-reference-macos/w...
Until we get rid of SNI[1] in HTTPS for good there will still be providers (like my ISP) that do deep packet inspection on SNI and kill the connection right away if you happen to visit a forbidden site (and this was western Europe, yesterday, on a site behind CloudFlare)
About getting rid of SNI... https://blog.cloudflare.com/encrypted-client-hello/ Been working on that also.
I wanted to link CF efforts on this also but somehow I forgot. Thanks for sharing and I really hope you are successful at this because what I experienced yesterday was really infuriating. Even if having everything behind a CDN to avoid ISP spying is still not the optimal solution, but at least is an improvement given what ISPs have already shown.
Part of the counter-argument that has been so prevalent on HN (most recently: [0]) is that when you prevent middlemen on your network from being able to see what website you're browsing, you're doing exactly that: preventing anyone, even a trusted network administrator, from being able to inspect traffic. I'm all for DoH and ECH since US ISPs have a history of inspecting and logging traffic, but it seems like there should be a way to manage the devices on your network besides being forced to set up MDM on everything.
Yeah, but that argument sounds like asking people to use “logmein” as a password so they don’t need to install MDM on everything.
Management of devices without authentication and authorization means anyone can do it. Which is the state of things today (for DNS).
I think you've nailed the core of it.
Managing traffic over your network and the devices on your network are very similar tasks that aim to accomplish very similar things. However, they are not equivalent tasks. Relying on traffic management to accomplish device management eventually runs into conflicts. These may stem from unmanaged devices, guest devices, unmanageable devices, or the consequences of the total lack of authentication and authorization.
Ultimately, managing traffic and managing devices are not tasks that replace one another.
> it seems like there should be a way to manage the devices on your network
Administering devices with network settings is convenient, but rapidly vanishing because there's no technical difference between you administering your local network and a totalitarian ISP administering their users.
My ways of dealing with the modern world, in order of preference:
1. Use Free software, so that devices develop user-empowering features instead of being locked down.
2. Firewall all general Internet access from a device/VM, and let it talk to local network devices only.
3. Firewall the device/VM from accessing most of your network, allow Internet access (ideally through a VPN), and inspect the hardware to make sure there aren't microphones or cameras.
The vast majority of users do not have a "trusted network administrator"; they have a hostile upstream network. That's where the defaults come from. Trusting the network (or anything outside of the device itself) should always be a non-default setting, and nothing from the network should be able to change that setting. You should have the option of configuring a device you own and control to make its traffic inspectable, but that should never be the default.
> a way to manage the devices on your network besides being forced to set up MDM on everything.
It's sort of like cleaning malware off of an infected PC from within the infected OS.
It was always theoretically impossible, and now we're just seeing the gap of "Well in this case the enemy was imperfect" closing. It was never going to stay open in the first place.
You can bypass SNI inspection [0] with tools like GreenTunnel [1] and Intra [2].
[0] https://twitter.com/vinifortuna/status/1304189371688660992
Thanks for the link, just tried Green Tunnel on the use case where my ISP is blocking me and just managed to change the error from PT_CONNECT_RESET_ERROR to PR_END_OF_FILE_ERROR.
Side note, looks like that if installed by snap on Ubuntu 20.10 it cannot automagically change the proxy configuration in Gnome
Enabling proxy manually makes it work but yet, it doesn't circumvent my ISP filtering :(green-tunnel:system-proxy [SYSTEM PROXY] error on SetProxy (Error: Command failed: gsettings set org.gnome.system.proxy mode manual green-tunnel:system-proxy /bin/sh: 1: gsettings: not foundDoes Intra work?
I can't find any source on intra working to prevent SNI sniffing. The page itself only mentions DNS, and Googling doesn't reveal any other source for that.
E: NVM, found it. It does like it uses split hellos.
The correct answer will be reverse, get rid of SNI completely and enforce ESNI everywhere.
Most bad entity now only need to block ESNI, and then the client will happily fallback to plain SNI.
If everyone enforce ESNI only, then it is not gonna going to work.
Just like nowadays, a browser can't view https site is completely useless because most of sites on internet were already encrypted(and the percentage is only going to be more) no matter how useful/useless the site is.
We don't need to kill regular SNI to fix that problem. If a site's DNS record indicates that it supports eSNI, and a connection with eSNI fails, then the browser should hard-fail. And middleboxes can't lie about whether a site supports eSNI, since that's protected by DNSSEC (and it should be coming over DoH anyway). This would break the bad actors without breaking every site that didn't upgrade to eSNI.
As long as plain SNI is still a option, bad actor will try to enforce you to use that. So they can do bad things.
China seems already done that and blocked esni. And the sites eventually gave up esni because people complaining they can't connect to it.
A deprecation likes that(ex. browsers nowaday marks every http site as unsafe) ensure it is not available to everyone. So some sort of these attacks never work.
No! Solving the SNI problem is far from enough.
The server IP address can be easily correlated with the domain for 90% of Internet traffic.
Do you have a citation for only 10% of internet traffic using CDNs? Even things like cloud load-balances and ephemeral IPs make those associations hard and we’re in third decade of major web properties using CDNs.
Use a VPN then.
Preventing the target resolver from seeing client's IP address breaks GeoDNS. This is already a problem with 1.1.1.1 which doesn't honour the EDNS client subnet extension.
Given generally DNS is just the start of an intereaction, usually followed by the connection directly between the client and intended destination, I don't see what kind of snooping these privacy measures are there to prevent.
Valid points, but...
> Preventing the target resolver from seeing client's IP address breaks GeoDNS.
If the proxy and the target are in the same metro as the user, it shouldn't really matter.
> This is already a problem with 1.1.1.1 which doesn't honour the EDNS client subnet extension.
1.1.1.1 runs at Cloudflare's edge. Most likely it is recursing DNS from more or less the same location as the user and so ECS isn't really required when in fact it exposes the client unnecessarily to upstream name-servers.
> I don't see what kind of snooping these privacy measures are there to prevent.
The one where DNS resolvers build to sell browsing profile of its users?
> If the proxy and the target are in the same metro as the user, it shouldn't really matter.
Having ran one of the largest public DNS resolvers on the internet, I can tell you it is a big problem. GeoIP providers do not have the fine grained data to be able to tell that a resolvers unicast address is in Seattle vs Chicago for example.
Cloudflare doesn't care about edns-client-subnet because the only downside is that other CDNs appear slower to their users.
Aren't these DNS resolvers largely the ISP anyway? They know where any packets are going anyway for each user. Seems to be a trivial hurdle to jump.
Encrypted DNS only solves hi-jacking, it doesn't provide privacy. DNS must be public. It is trivial to run a DNS server to build a simple reverse lookup table. This is as much privacy as the TSA provides airline security.
> I don't see what kind of snooping these privacy measures are there to prevent.
The point of this is to prevent some cloudflare competitor offering DoH, but logging what dns names each client looks up, and selling that information, or using it internally.
Think about the ways that facebook would abuse that information if facebook ran a popular DoH resolver. For example, they detect that you have used a hookup app (based on dns lookups for their servers), and boom, now your facebook feed is full off condom adverts. Or thousands of other scenarios, some even more creepy than that.
As I see this, this is a very clever move by Cloudflare.
It's intentional to force websites to move to their CDN or atleast use a CDN with anycast and prevent you from making your own CDN like you could cheaply before (spinning up DO droplets and doing loadbalancing with geo DNS).
That's a weird take. (a) this is a proposed standard not just some Cloudflare service and (b) you can just use Cloudflare DNS if you want and forget about the rest.
It'd have been fabulous if Cloudflare ran ODoH Proxy too.
Thus increasing the cloudflare value-prop of anycast based load balancing.
The DNS server is centralized storage of all your browsing habits.
> which doesn't honour the EDNS client subnet extension.
background: https://news.ycombinator.com/item?id=19828702
GeoDNS was always a "works most of the time" hack relying on some widespread (but not universal) implementation details in routing and DNS infrastructure, no?
Interesting that apple is increasing its stake in privacy. On all their billboards and advertisements of course they like to present it as a boon to the customer. More importantly, I think it’s a negative for personal data hungry competitors while being relatively unrelated to Apples business
Yup I just see this as an alignment of interests. In this case, Apple's interest happens to align with that of their consumers.
And I for one am happy that they have taken up this cause and put their weight behind it, whatever their intentions may be, the effect is that it makes the web more private for those of us who deem it important to move away from the "monetizing data" cancer that has spread all over the internet.
When you need a log-log plot to make the performance degradation not look so severe, you have issues...
145 ms response time when DNScrypt is around 10-25 ms, and anonimized DNScrypt is around 2-3x, wow
If anyone wants the draft RFC:
https://tools.ietf.org/html/draft-pauly-dprive-oblivious-doh...
Do I understand this correctly that if DoH is implemented, none of the firewalls will be able to block the web sites? Including the pi-hole firewalls, as an example. If that's the case, this situation can't stand for long. Does this meant that the DoH would need to be extended to allow firewalls to decrypt it?
If not, here is a PaloAlto Networks blog advertising capability to block all DoH traffic, presumably at work [0]. It looks like you might not be able to use DoH at work, the way it currently stands. I wonder what would be the right solution?
[0] https://live.paloaltonetworks.com/t5/blogs/protecting-organi...
What's stopping your PiHole from just getting its own HTTPS cert?
I am still a noob to the networking. I just started using OpenWrt router to manage my home network. OpenWrt has an option to enable DNS over HTTPS. Is this all one would need to do in order for the firewall rules still work with the DoH browsers?
>Do I understand this correctly that if...
You understand correctly. DoH mostly defeats pihole and the like. Presumably Google and other ad companies love this.
DoH isn't blocking anything that any existing application can already bypass. It merely points out the fact that the rise in encryption makes it difficult for third parties to administer the software running your device.
DoH will not be extended to allow firewalls to decrypt it. The entire point of DoH is that firewalls cannot do that. It's not hard for an administrator to set up their own DoH server, though, so there is no need as long as that server can be configured at the network level. This is currently possible through group policies on Windows and most MDM applications on mobile devices.
The nice thing about encryption is that only the software and the server know what is being exchanged. The not-so-nice thing about encryption is that only the software and the server know what is being exchanged.
You already can't see the difference between your computer connecting to some S3 bucket to download a picture of a cat or malware connecting to a S3 bucket to get a list of IP addresses mapped to host names. It's possible to run fully-fledged VPNs that look exactly like normal HTTPS traffic on the network level. DoH changes nothing about that.
On enterprise networks, IT administrators can already put measures into place that prevent DoH on support applications and restrict HTTPS connections to trusted hosts (through SNI sniffing + validating the connection). The upcoming encrypted SNI will make this harder, but group policy will be able to disable eSNI on any trusted computers and browsers, leading only untrusted software to be unverifiable, which can then be reasonably classified as malicious.
If IT administrators cannot put measures on employees' devices then... well, they've already lost. Before DoH, they just didn't know that they'd lost yet.
Palo Alto currently blocks DoH by decrypting all TLS traffic (ranging from cookie recipes to naked pictures your phone is syncing to the cloud) with a man-in-the-middle attack which requires their certificate authority to be installed on your system. They filter out the DNS requests inside those TLS streams and apply some kind of rule to them. Right now, they can either block the requests or change the contents (as long as the website and/or client doesn't use DNSSEC).
With ODoH, changing the contents will no longer be practical, but blocking the connection will be. Together with decent group policy settings, networks with such a setup should be as safe as they were without ODoH.
On a side note: these types of middleboxes are often trying to be transparent and silently violate many standards, breaking stuff that works on every normal network. Stuff like these firewalls are the reason that TLS 1.3 is pretending to be a special type of TLS 1.2 every time it connects, because otherwise some big middlebox vendors' platforms freak out and kill the connection. I doubt the inevitable transition towards HTTP/3 or even HTTP/4 will do these boxes much good. I don't put a lot of faith in devices like this.
I understand why Cloudflare wants this (marketing, as well as being able to serve their customer’s content through restrictions, thus making them more valuable to those customers),
but why does Apple want this?
My knee-jerk is that they want to further hide/make unstoppable things like the Gatekeeper network checks, but there has to be more right?
2 reasons I can think of.
Firstly, Apple loves to act like they are always taking your privacy very seriously (of course that's not always true), so for the cost of a few engineers, they get a massive marketing point. "We take your privacy so seriously that we developed a new protocol to do so"
Secondly, Apple has an awful case of NIH syndrome. If they didn't develop it themselves, they would rather develop it from scratch themselves
Secondly, Apple has an awful case of NIH syndrome. If they didn't develop it themselves, they would rather develop it from scratch themselves
Not when it comes to security and privacy. They knew DoH and DoT were good, but not good enough when it comes to privacy, which explains why they didn't just implement it like Google and Firefox did.
Instead, the worked with Cloudflare to standardize something that's better.
It is part of their marketing. The fact that others sell or actively use your data, e.g. google, facebook, microsoft, apple was handed the opportunity to charge a premium for the absence of such tracking and data usage. If you watch the presentations, they branded/poised themselves as the privacy centric approach.
I see your point about how they (will) position it, but I’m still curious about their actual motives.
As a preface, I am about to describe something I personally view as absolutely horrific. Please, dear reader, in no way interpret these comments as approving.
Privacy is a luxury. It's something rich people buy and poor people can't afford to concern themselves with. Apple sells a luxury product, and has no particular stake in invading customer privacy at the moment (iAd was never successful enough to change that). So they add a feature to their status symbol to address the concerns of their customer base and work with a partner company that can actually deploy it.
Exactly that. It strengthens their image amidst the whole ordeal with app signatures. It is equivalent to investing for an ad.
Why would Apple care about hiding Gatekeeper traffic from internet providers?
I’m guessing they want to hide it more from users. The recent bypassing of local firewalls shows this, for example.
The bypassing has nothing to do with wanting to hide it from users Even if they did care about hiding it, they know how trivial it would be to discover it, as we already saw only hours after the release.
And those lookups have nothing to do with DNS, so this wouldn’t help nor hurt anything related to that.
The bypassing has to do with exerting their control despite user wishes. Hiding “complexity” from users is one method that is at the core of Apple’s brand.
Yes, very smart people uncover this kind of thing regularly, but the trend feels like Apple is just trying to refine the process until they have a “perfectly secure” device by virtue of the fact that not even legitimate owners are able to enforce their wishes when those wishes are counter to Apple’s mandates.
You’re making assumptions about their motivations, and they’re not correct. They are not doing it despite user wishes. They did it under the reasonable assumption that the user has no such wish. It likely didn’t cross their mind.
Yes. We’re both making assumptions.
Apple is surely aware of Little Snitch and other firewalls, and that the markets for that are dependant on a percentage of users who want absolute insight and control in to their network traffic. Similarly, there are journalists and sources who must by nature be very cautious about all network traffic. It would be hard to argue I think that Apple is unaware of both of these groups of users, and if they are aware; it must follow that it crossed their mind.
Whether that crossing their mind means they discarded it or intentionally chose to go against it may be a question that only gets answered in hindsight since Apple says very little publicly.
Because it's none of Comcast's business what software I run?
Because it's none of Comcast's business what software I run?
There's no way for your ISP to know what software you're running.
Gatekeeper checks if your app is malware (or not) and if its been signed with a valid Apple developer certificate. The OCSP look up goes over in the clear currently, but that's how OCSP works everywhere. Your DNS provider can see the OCSP lookup but that's about it.
Apple is in the process of addressing this; you can read the details of how the current process works at https://eclecticlight.co/2020/11/16/checks-on-executable-cod...
For most of the software on my computer, the developer certificate is enough information to know what software I'm running.
Are they going to think I'm running some other piece of software signed by Slack Inc.?
For most of the software on my computer, the developer certificate is enough information to know what software I'm running.
All your ISP can see is certificate hashes, OCSP lookups and DNS queries. It can't know what certificate hash is connected to what developer application…
Presumably that's an unsalted hash so that it can be checked against the list of certificate revocations, so whether it's a hash or not doesn't do anything for privacy. It's the same hash of Slack's dev certificate that every other Slack customer is sending.
Anyone snooping the connection can figure that out and see that my computer said "Check the revocation status of Slack Inc.," and the same goes for literally every other software company's certificate hash.
I'm glad it's being fixed but it's still bad that it was done this way in the first place.
It's not hard to match up a certificate hash to the issuer, because most issuers will likely only have a couple of certificates to simplify internal PKI. It's something that can be solved with a rainbow table, there aren't even salts involved.
It's not hard to match up a certificate hash to the issuer, because most issuers will likely only have a couple of certificates to simplify internal PKI.
These are Apple certificates; they have nothing to do with a company's internal PKI.
It's something that can be solved with a rainbow table, there aren't even salts involved.
1. Certificates change; probably yearly, knowing Apple.
2. The OCSP check get cached; the certificate lookup doesn't happen every time you launch an app.
3. You can block the OCSP lookup if you're all bent out of shape about it or strip the developer's signature and sign it using a different certificate.
4. The new protocol for checking will be encrypted and there will be UI for opting out of these checks.
Metadata privacy is very hard to solve and traffic analysis of non-Tor traffic is pretty accurate, which is also applicable to CDN traffic regardless of how well DNS is protected.
One should also note that, even if you use ODoH, eSNI and even Tor (or any VPN service), your ISP could still reliably fingerprint your web access activity at the source using deep learning with over 96% accuracy as shown in this study (https://distrinet.cs.kuleuven.be/software/tor-wf-dl/).
So while ODoH is a good thing (and also recommended in this study which has shown the weaknesses of DoH/DoT https://www.esat.kuleuven.be/cosic/publications/article-3153...) and is very similar to DNS over Tor with a DNS hidden service resolver (which Cloudflare also provides). It won't prevent a skilled and motivated adversary from determining your activity and possibly apply censorship.
I would guess that a solution to mitigate these would be to use an hybrid solution of VPN over Tor (or Tor over VPN) while also using DNS over Tor or ODoH and eSNI.
Even better, IMO, would be if all targets were also proxies and a client could choose -- at "query time" -- any combination of (proxy, target) that they prefer.
If you wanted to go a step further, you can even allow "chaining" of proxies, such that the path a query takes might be, in an extreme example, similar to how Tor operates:
Client -> Proxy 1 -> Proxy 2 -> Proxy 3 -> Target -> Resolver
Anyways, this is kinda sorta interesting, I guess, but honestly I'm more excited by and looking forward to the (hopefully!) eventual adoption and roll-out of "DNS SVCB and HTTPS RRs" [0] -- one of the other I-Ds (linked in the OP) on which ODoH is built -- and I suspect many other HN'ers will be as well (although I'd happily settle for SRV RR support in browsers).
--
[0]: https://tools.ietf.org/html/draft-ietf-dnsop-svcb-https-02
Sounds like Tor/Onion Routing?
Why encrypt the first hop? Why isn't this just plain DoH with a simple CONNECT forward proxy to 1.1.1.1, like Signal's Giphy proxy [1]?
Answering my own question.. A cryptographer friend offered an answer to this question: The network operator may be the same as or colluding with the target resolver, defeating the anonymization of the proxy.
Once we say we need encryption on the first hop, then I can see the logic in using a stateless protocol instead of TLS for the second hop, to avoid TLS-in-TLS and all the round trips associated with that.
Side note: It'd be cool if these new protocols used the more generic Noise Protocol Framework [1] instead of a custom, more specialized protocol they just came up with like HPKE [2].
[1] http://noiseprotocol.org/noise.html [2] https://www.ietf.org/id/draft-irtf-cfrg-hpke-06.txt
Totally agree. Just posted my own question here: https://www.pcwrt.com/2020/12/oblivious-dns-over-https-vs-do...
I'm wondering how they still get good performance with a proxy server in between, the plots seem quite close to each other (maybe because logarithmic?).
Also, not sure how useful the Tor comparison is, since Tor does 3 hops as opposed to their 1 so it would be a shame if it doesn't beat that.
Serious question, why do we need ODoH at all? Isn't a plain proxy good enough to achieve this? https://www.pcwrt.com/2020/12/oblivious-dns-over-https-vs-do...
So Google got sued by ISPs which lobbied an investigation by DOJ for trying to encrypt DNS: https://www.engadget.com/2019-09-29-congress-doj-scrutinze-g...
Will ISPs be too scared to sue Apple and Cloudflare for this? Or are they giving them an out?
If I understand this correctly this was mostly about Google getting an unfair advantage over the ISPs.
Which wouldn't be the case if everyone loses access to the IP + DNS request info.
The basic idea makes sense to me and it's great to see efforts to improve DNS privacy. However, I'm not really convinced by Cloudflare's analysis of the processing overhead:
The blog post only discusses how the proxying and encryption affect latency but not the processing at the server. In contrast to plain DoH (or DoT), where only symmetric cryptography is used after the first set-up, ODoH requires asymmetric cryptography (which is several orders of magnitude slower) for each individual request. The "less than 1ms" that they claim for the 99th percentile is no problem for the client but it is a problem for the resolver. Asymmetric cryptography is also used for verifying DNSSEC responses, but this is only necessary for records that are not cached.
On the other hand, an ODoH resolver may require to set up and keep track of a lower number of TLS connections as the number of proxies is likely smaller than the number of clients.
I suspect that practical matters will interfere with widespread adoption of encrypted DNS.
In my state, Comcast is going to start charging heavy bandwidth users extra. After a few people get surprise bills, I suspect that lawmakers will require that internet providers break down a bill by application.
Mobile OSes already support per-application network usage breakdowns. This is an OS-level feature, and not an ISP-level feature.
I’d just get Starlink. Even if the deal was worse in terms of cost it would be a way to say fuck you to the ISP. Without some way to do that ISPs will not be able to get away with such customer hostile behavior.
> Without some way to do that ISPs will not be able to get away with such customer hostile behavior.
I guess it won't take long until the first community or HOA decides to ban Starlink dish installations for faked "optical nuisance" issues.
The fcc will have something to say about that. They've already banned rules against antennas and sat dishes for TV.
Faked? What other reason would an HOA have to ban them?
A cable company offers to wire the neighborhood saving the builder some money. In exchange the HOA setup by the builder bans satellite dishes. In more annoying cases the HOA negotiates a group plan included in your HOA assessment bill; that really stifles competition because paying for an alternative on top of cable is irrational. The FCC banned satellite dish bans back in the 90s.
Apartment building operators still do this scam. The incumbent cable company will share revenue in exchange for the landlord refusing any competitor wiring.
Rent seeking in ISP markets is peanuts compared with zoning though. If real estate was much less supply constrained landlords would never fathom annoying a customer with inferior utilities.
After seeing similar things throughout the years, I feel like that is very doubtful. When tested, the push for privacy is much stronger than the push for cost.
I'm surprised to see Cloudflare and Apple collaborating on privacy.
What does Cloudflare think of Safari's new CNAME-cloaking detection to block cookies? https://webkit.org/blog/11338/cname-cloaking-and-bounce-trac...
The reason I ask is because Cloudflare's "orange cloud" DNS mitigates that protection because it prevents Safari from detecting the cloak. On the other hand, I haven't run into many engineers who think CNAME-cloaking actually hurts privacy in light of Safari's other efforts to partition local storage.
Does Cloudflare think it would be help privacy for Apple to know the final IPs behind orange cloud DNS?
I would love to see Firefox be an early adopter of this, regain market share and save us all from Chrome!
So, having read the blog post from Cloudflare I don't understand why the proxy (needs to terminate|terminates) TLS.
I thought HTTPS proxying (or rather: Any TCP protocol) was a solved problem by the HTTP CONNECT verb or SOCKS proxies.
What am I missing?
The user's IP address is masqueraded by the proxy, and neither the DNS mothership (Cloudflare) nor the ISP get to see both who the user is and what they requested. It's an extremely desirable property DoH currently lacks
Yes, I understand that. But I don't understand what ODoH does better than a run of the mill SOCKS proxy, such as Tor.
Tor is not a run of the mill SOCKS proxy, not least in that it inserts arbitrarily high latency into the user data path. On the other hand, an actual run of the mill SOCKS proxy would have visibility of the user's queries and their identity, defeating the purpose of the design.
> an actual run of the mill SOCKS proxy would have visibility of the user's queries and their identity, defeating the purpose of the design.
Why would it have visibility of the queries? If I send a TLS connection (containing my DoH query) through that SOCKS proxy, then the SOCKS proxy is unable to decrypt that TLS connection without breaking certificate verification and thus can't read my DoH query.
Very good point! Sorry, I was confusing myself thinking about classic DNS.
Abuse. The message must be a DNS query, not arbitrary tor traffic.
In a nutshell: client encrypts to proxy, which decrypts & removes client info, then asks resolver.
> “What ODoH is meant to do is separate the information about who is making the query and what the query is,” said Nick Sullivan, Cloudflare’s head of research.
> In other words, ODoH ensures that only the proxy knows the identity of the internet user and that the DNS resolver only knows the website being requested. Sullivan said that page loading times on ODoH are “practically indistinguishable” from DoH and shouldn’t cause any significant changes to browsing speed.
> client encrypts to proxy, which decrypts & removes client info
This is incorrect; the proxy doesn't decrypt. It just proxies. From https://blog.cloudflare.com/oblivious-dns/ :
> The target decrypts queries encrypted by the client, via a proxy. Similarly, the target encrypts responses and returns them to the proxy. [...] The proxy does as a proxy is supposed to do, in that it forwards messages between client and target.
Thanks for the clarification. So client encrypts for the resolver and sends an opaque payload to the proxy.
All security theatre while SNI is still universally deployed. Even then most IP blocks are static and easily correlated to source site.
A tor-like solution is the only real solution for this threat model
So I do wonder how such systems can be designed or implemented such that geoip systems can still work.
While I'm sure aws route53 and cloudflare's own routing systems can handle this properly, Cloud isn't quite the answer. Not every workload fits on the cloud (see: Discord, which runs on leased servers), and a system that breaks down if your rented datacenters aren't in alignment with Cloud operating regions doesn't make a great solution.
> Not every workload fits on the cloud (see: Discord, which runs on leased servers)
As far as I'm aware (don't work there), only bandwidth/CPU-heavy stuff like voice and video live on rented dedis; the core chat services live in GCP.
Discord already doesn't need geoip, since they just direct you at wherever the server location (that you chose) is
At what point should we just throw out IP out of the window and figure out something new ? OK maybe not IP since all hardware infrastructure is based on it, but the whole idea of associating services to publicly open ports on the target machine. I'm thinking connections should be encrypted at the operating system level and then services would plug in at some higher level in a way that cannot be detected by outside observers.
What's the advantage of this over specifying a DoH provider (as we do today with plain DNS)?
Unfortunately I suppose the only way to really do that is with a resolv file (adlist/blocklist) of DoH hosts (which exist) but instead of pointing to 0.0.0.0, point to <preferred DoH>.
Edit - d'oh! I see it now - that would mean DoH provider knows query and IP, whereas here the ODoH proxy knows your IP but not the query. Nice.
Why not DoT? And DoH is mum on http cookies: "Determining whether or not a DoH implementation requires HTTP cookie support is particularly important because HTTP cookies are the primary state tracking mechanism in HTTP." https://tools.ietf.org/html/rfc8484
Is it not still possible to do a pi-hole kind of setup for DoH or ODoH? All you have to do is setup the server as a proxy for all http(s) connections on top of DNS connections and trust its cert on the client. If we can reliably block all ad networks with uBlock origin, picking out DNS requests from other http requests should be even simpler, right?
DNS privacy for DoH effectively means we all lose the ability to control what our devices are connecting to. In particular, we can't block ads and trackers at the network level. The lack of fallback to regular DNS in the spec means we will choose between devices that track us while they work, or devices that are broken.
No discussion of DNS privacy should go without a link to Bert Hubert's awesome talk on the subject: https://www.youtube.com/watch?v=pjin3nv8jAo
> ODoH ensures that only the proxy knows the identity of the internet user and that the DNS resolver only knows the website being requested
Who is the proxy here, and who the DNS resolver?
From the CloudFlare blog post:
> A key component of ODoH is a proxy that is disjoint from the target resolver. Today, we’re launching ODoH with several leading proxy partners, including: PCCW, SURF, and Equinix.
This is a protocol, not a product. There is no set proxy, or resolver.
The title of the article is really misleading. I though of a succesor to IPv6 and not DNS. It shouldn't say "internet protocol", thats technically not correct
> Cloudflare is committed to end-user privacy.
Pretty crucial hyphen
Are ODOH resolvers by any disjoint partner available yet? The only one I see is the CF-owned and run one.
> Sullivan said a few partner organizations are already running proxies, allowing for early adopters to begin using the technology through Cloudflare’s existing 1.1.1.1 DNS resolver.
In other words, in order to thwart efforts to make the internet anonymous , US companies are planning to takeover DNS for the vast majority of people.
Oh, please. ODoH is a proposed standard. Use whatever the hell proxy/resolver you feel like, wherever you like.
DNS is a shit show of unencrypted data flying around being scooped up by God-knows-who and along comes someone proposing a standard to fix said shit show and this is the response people get.
Would have been a million times better if cloudflare had used it's considerable clout and resources to push/lobby for widescale DoT adoption rather than this fucking Frankenstein shit show that is DoH (now with even more garbage attached to it in the form of a proxy)
Decentralized spying > centralized spying
Choosing to have google or quad9 spy on you doesn't mean AT&T no longer gets your DNS data when you use plain port 53.
I’m good with the Apple’s privacy-oriented stance. But I can’t stop to think what will happen when advertisers knock on Apple’s door trying to get their hands on the users’ data that one else can access. Is Apple going to sell it out for more profits?
The whole design of this DNS system would mean that even if apple ran a ODOH proxy, they still wouldn't be able to see what the request was for.
What data can apple give them?
Apple almost never bends over for advertisers. The closest they have done is pushing a privacy change launch date further down the line.
https://www.theverge.com/2020/9/3/21420176/apple-ios-14-trac...
https://www.telegraph.co.uk/technology/2020/12/08/advertiser...
It's just marketing. Apple has already shown they will sell you out with PRISM.
Who knows what other backroom deals are happening outside our knowledge. The only reason we found out about PRISM is because the gigantic scale and Snowden sacrificed Everything to let it be known.
I think there's a big difference between selling data for profit and the government literally forcing you to give up data based on national security laws or else forcing you to close your business. There's almost nothing Apple can do about the latter case (or any other company for that matter).
Didn't twitter survive?
Also how would we know if Apple is working with other companies? It's not like they are known to be transparent or Truthful.
Also how would we know if Apple is working with other companies? It's not like they are known to be transparent or Truthful.
Apple puts privacy and security front and center as part of their brand. They zig while everyone else is zaging, trying to make a buck on user data, which they don't do.
For starters, here's their transparency report: https://www.apple.com/legal/transparency/
One thing I like to remind people of is the fact that the Snowden docs that revealed PRISM were years old at the time of Snowden gathering them, and even older at release... just imagine how much further things have progressed in the 10+ years since. (iirc lots of them had 2007 dates on them)
Hilariously I see privacy invading advertisers loving this. No more DNS blocking ad traffic! And since it's only a matter of time before Apple removes root access on their PCs, it puts them in complete control off what you see.
Can the proxies be (ab)used to proxy arbitrary HTTPS traffic?
This seems to require DNSSEC as a key function. @tptacek ?
It has nothing to do with DNSSEC.
Huh? They say this:
> The whole process begins with clients that encrypt their query for the target using HPKE. Clients obtain the target’s public key via DNS, where it is bundled into a HTTPS resource record and protected by DNSSEC. When the TTL for this key expires, clients request a new copy of the key as needed (just as they would for an A/AAAA record when that record’s TTL expires). The usage of a target’s DNSSEC-validated public key guarantees that only the intended target can decrypt the query and encrypt a response (answer).
So this looks like relies on DNSSEC as a core part of its security, and that any resolvers willing to participate in this protocol would have to set up one.
Oh, gross! I missed that; I read the TechCrunch article and thought I understand what they were going for. Thanks for the correction. That's disgusting.
Thank you for the part regarding DNSSEC--I was just about to post the same thing.
This should be called DOHW, DNS over http made worse.
The more these big corporations involves in this process, the more we are gonna lose our privacy.
Centralization and too much power in certain amount of hands are the source of all evil.
Has Google produced any similar initiatives?
Probably better source, the blog post at Cloudflare: https://blog.cloudflare.com/oblivious-dns/
Thanks. The original post redirects to:
https://guce.advertising.com/collectIdentifiers?sessionId=3_...
Which is blocked at the DNS level on my network.
As it should be.
Thanks! we've changed to that from https://techcrunch.com/2020/12/08/cloudflare-and-apple-desig....
REMINDER: Research proves that it's easy to correlate IP addresses in HTTP[S] connections with the domain you are connecting to with a very high success rate.
You can resolve the websites from the Alexa top 100k list and create a ipaddr -> website map that will successfully apply to 90% of Internet traffic without ambiguity.
A lot of research papers also show how easy it is to fingerprint and detect a TLS handshake.
Assuming the SNI problem is going to be solved, the other problems are still here.
TL;DR: use Tor.
Sounds promising. Get back to me when it’s gotten to the RFC stage. A ready-made solution thrown over the wall like this, is rarely what is ultimately adopted.
Whats the point?
Governments subpoena the information or just block the protocol outright. ( or in China, get it delivered to their door by Apple )
Commercial parties have a bag full of tricks from fingerprinting to embeds on the page itself to track you.
Privacy seeking users are already tunneling their traffic.
That leaves script kiddies at Internet cafes. TLS kind of fixed that already so... Good work?
As it stated in the article, ISPs tracking and selling the data.
Exactly that, no more, no less.
You, the proxy, and the DNS service, can be in 3 different countries. It's not bullet proof but makes it quite hard for a single government. Unless you are a Bond villain I think this is more than you need.
If you need more than that use ToR or similar.
While I agree that it's a step forward you should proxy your whole traffic anyways if you want to enjoy the benefits of encrypted DNS. Otherwise intermediaries routing your packets can still see what you connect to by looking at the IP header (or SNI, if not encrypted).
DoH/DoT is still useful because it allows you to proxy your DNS over Tor (for example) without having to worry about tampering (or surveillance if you also use separate circuits per domain).
Misleading title. Apple devices are not anywhere near ready to utilize this dns protocol. Apart from that, yeah let's shift our dns trust to one of the biggest data resolvers! The irony...
Encrypted dns might be already in use by government or military agencies, but they know too well the effects of cascading this tech down to the masses. They will never let this reach the public.
Apple devices are not anywhere near ready to utilize this dns protocol.
The latest versions of macOS and iOS already support DoH and DoT; Apple could push an update tomorrow to enable ODoH tomorrow if they wanted to.
Encrypted dns might be already in use by government or military agencies, but they know too well the effects of cascading this tech down to the masses. They will never let this reach the public.
You do know we've had encrypted DNS for years, right? It has some issues, which this new protocol is designed to address. There's no reason to believe "they" can or will intervene to stop ODoH.
I don't think you understand how DNS works.
DoT and DoH should not be confused for encrypted DNS.
Encrypted dns is still a myth to most users. Major resolvers do not support it since it directly conflicts with with their data collection business.
All forms of Internet communications can be largely encrypted. Dns is the last frontier remaining. It remains so for good reason...
I don't think you understand how DNS works. I don't think you're in a position to comment on what I do or don't know about DNS.
Encrypted dns is still a myth to most users. Major resolvers do not support it since it directly conflicts with with their data collection business.
Except those users using Firefox or Chrome, which come with DNS over HTTPS (DoH) preconfigured. Or those who've been running DoT on their home networks, which I setup quite a while ago now.
From the Wikipedia article on DoH, emphasis mine: "A goal of the method is to increase user privacy and security by preventing eavesdropping and manipulation of DNS data by man-in-the-middle attacks[1] by using the HTTPS protocol to encrypt the data between the DoH client and the DoH-based DNS resolver.
DNS over TLS (DoT) RFC: "This document describes the use of Transport Layer Security (TLS) to provide privacy for DNS. Encryption provided by TLS eliminates opportunities for eavesdropping and on-path tampering with DNS queries in the network, such as discussed in RFC 7626."
The lack of DNS encryption isn't what Apple and Cloudflare are addressing; it's that whoever runs the DNS resolver can still see the websites you're visiting and ODoH fixes that.
Again you keep referring to DoT and DoH which I insist do not encrypt your dns queries from your ISP. They may offer added security but do not keep your requests private. ODoH attempts to keep your requests private from the resolver only. A benefit which is a good step but doesn't ultimately keep your dns private from your ISP.
This is the major flaw I find with such claims of encrypted dns. Your isp can still see which sites you visit, oDoH or not.
So if your DNS request is encrypted to the resolver, and from the resolver to a second resolver (first resolver is ODOH proxy), then is unencrypted from that resolver to the authoritative nameserver, where does the ISP get to see your DNS query? Unless you mean SNI, which is its own thing being worked on[0] (yes, it only works for big CDNs to look benign and you probably could still correlate IPs via traffic patterns. Doesn't mean it'll be as easy as it is now, though).
No hubris here at all.
But seriously, fuck this protocol and fuck every other BigCorp-sponsored protocol to remake the Internet. We the People Who Implement Protocols are too busy keeping the lights on to chase incremental, nice-to-have improvements.
Would you believe that HTTP/1.1 was made by people from Microsoft, Xerox, and Compaq with the sole purpose of being a nice-to-have improvement over an existing widely-used protocol, HTTP/1.0?
I urge people to stop repeating Apple Advertising. Claims of privacy and security are debunked weekly. You put yourself at risk if you believe it.
Do you have actual proof of this or are you just going to make misleading claims yourself?
Privacy is a buzzword to boost sales even more. Perhaps the biggest problem with Apple is its nasty monopoly strategies, remember the times you could easily add more RAM, change batteries?
This buzzword is actually useful though. It gave us ESNI, Cambridge Analytica, antimonopoly memes and whatnot.
Your statement is 100% correct but misses the entire point
https://nibblestew.blogspot.com/2020/04/your-statement-is-10...
Debunked where? If they were I’d expect HN to be the first to publish