Apple and Google Monitor Notifications. We Need Push Notification Alternatives
tuta.comI campaigned for this for years when working on fuchsia, to folks in fuchsia and android folks.
We lack a unixy API for “the radio is now on, do your reconnects or send your queues”
The platforms always hoist this stuff higher level and it rarely works that well.
Platform leads insist the platform will do it better, but it’s never true. They also insist that persistent connections are battery killers - which for sure they _can be_ but done properly (and with the aforementioned api) it can work just fine.
Establishing such an API in the Linux and BSD ecosystem would make a good step toward encouraging its exposure in the places we need it.
The Play Services people would have a stronger case if FB Messenger hadn't outperformed them on so many metrics of this type for years. One of the shocks of Copperhead/Graphene is how the absence of the play service push mechanism radically improves your battery life.
It is definitely the case that doing such things efficiently, especially in the world of evolving radio technology, is beyond the absolutely overwhelming majority of mobile app developers, and apparently in many cases even very good ones. It also seems unlikely that having the platform poll a process to then do the push (as your API would, in response to something like a radio burst about to be made) is a good idea, as keeping the app process asleep if at all possible is distinctly preferable, and most devs struggle with what is a real time constraint. An API to achieve this is definitely buildable though, it does require offloading a certain amount to the platform.
The truth is centralized services are strategic for the platform owner these days, so any technical argument for them will really be on that basis.
I also had some experience in that space, and my conclusion was that what was really hard is handling keep-alive correctly. Some mobile networks will very aggressively flush their NAT entries while other will be fine with a packet every hour or so. And sometimes the NAT timeout changes depending on the server's IP range. Building a solution that is both battery-optimal and that will keep the connection alive is pretty hard.
QUIC (or HTTP/3 with enough low-level control) sounds like a good fit: there is no NAT connection that has to stay alive, but the server can remember the handshake for a quick restart. Doesn't matter if you switch between wifi and LTE. When ever the app is told radio is now available, send a single UDP packet to request new notifications, server sends one of more UDP packets back in response.
But how would google get all the metadata if the app just connects to its own server to know about notifications?
systemd-networkd and NetworkManager have something to that effect.
There’s however a big difference between an interface coming online, an interface getting its addresses, a particular address being resolvable and reachable, and the particular address being routed to the place you actually want to reach. In an ideal world, you want all four; NM kinda sorta has it. There are d-bus APIs, just maybe not easy to use in shell scripts CLI tools.
The routing table won't get flushed in between the slumber of the WiFi or 5G modem. You do not have to care about IPs or interfaces being up - we just need to synchronize network work so that most comms can be done in short bursts and then everything can go back to sleep. iOS does this, has done this for years.
Can't you hook into NM and provide the missing bits with scripts? Or do you need also to listen to "something else" to get the events like "the modem just went to sleep" which is not quite the same as deconfiguring it?
(I think no wonder those pieces are missing, it usually isn't happening on computer-shaped things, more on embedded- and phone-shaped things.)
This irked me recently. I think it’s very reasonable for networkd to delegate everything after the interface meets an “online criteria” to later services. However I was disappointed that you couldn’t then tell networkd an interface was “active” (vs. online) later down the line.
Would be interested to hear how you & others approach this. Perhaps I’m not thinking systemd-y enough hah.
Can you describe your use case in a bit more detail? (Also to make sure it’s not really a XY problem)
systemd-networkd sure keeps some state tracking. There are systems where it’s the only thing managing the network, so it doesn’t have anything to delegate to. You can see it by running networkctl before and after yanking the Ethernet cable.
Even if you had that on Android, they won't allow you to open a network connection in the background if the device is sleeping. You still need to send messages through FIrebase.
It's disappointing to see everyone freaking out about this while ignoring the hundreds of different analytics SDKs littering every single mainstream app out there that leak even more metadata and store it on the provider's server (unlike push notifications which can be end-to-end encrypted).
One doesn't exclude the other - we regularly criticize tracking code in general.
Those who take the high road and claim high standards are always easier to attack when they slip. With everyone else, there are always going to be shrugs and expectations of incompetence and indifference.
There is no way around it: “everyone freaking out” is actually powered by attention seeking algorithms taking shots at what works.
Apple has taken a very strong stance on privacy. This forces everyone else to keep up. But without any understanding of technical complexity, most customers only really have faith and marketing to go off. These headlines deliver serious blows to that image.
I think this is great news for the rest of us. We should be weary of every detail of how these devices work. And if they’re claiming to take the high road, we should be paying attention closely when they’re not.
Criticise it all…
Man I went to a supermarket website the other day and received 87 cookies for free
If only the supermarket would give me 87 cookies when I visited. :(
Most of the cookies in supermarkets are not very tasty because they are optimized for low cost and long storage time.
The cookies that don't come from the store's bakery do tend to be dry, but they are well loved. For example, you can find faithful duplicates of any variety of Girl Scout cookie.
The cookies that do come from the store bakery don't even match your description.
I'm using Zoho as my main email hosting provider. Can I use tuta as the client? Should I?
I like Zoho stability and pricing. And, the client is somewhat poor compared to Gmail, to be honest. I like the scheduled send and snooze options of Gmail, but cannot figure out if Zoho provides similar functionality.
What is the best client for non-Gmail? Scheduled-send and snooze are probably server side functionality, so it isn't just the client. But, there must be a better client than the Zoho one!
I use Zoho for my client's email host and I like it for that, but I try not to use any of its clients except for necessary configuration.
I configure Zoho into my clients devices for them in their phone mail app (with iPhone I use EAS so they get the email instantly with push). For my personal Zoho addresses (admin stuff) I have them send to my FastMail account. I really like FastMail's web app and Android app. So much better than Gmail and I have had only one or two minor issues with the Android app, usually when I have a spotty connection.
Thanks so much for you comment. I'm very intrigued.
I'm not sure I understand which clients and which services you use here. You use Fastmail, which is a paid service for email, right? But it sounds like you also use Zoho services as well. Why both?
Zoho is so cheap: I pay $12/yr for multiple domains and multiple email addresses on each domain. It was a little tricky to validate the domains and get them added, but it's been smooth sailing ever since.
Should I add Fastmail in the mix? What does that add here?
Yup, correct, I use FastMail and Zoho. I used Zoho for my clients' emails because it's just so cheap to add extra users. $1/mo/user. Which makes it easy to resell to clients and not worry about margin. I just found more personal preference for FastMail for managing my own stuff.
I have about 5 domain emails (different domains), 3 Gmails (down from 10!), and some from from Zoho I all manage in FastMail. It was a breathe of fresh air after leaving GSuite. I pay for the $50/yr plan ($5/mo monthly).
Hard to say what FastMail would add for you. It checked my boxes at the time I was switching from GSuite/GMail but it has been a quality of life improvement for sure
> Final thought: Every user should be able to choose a "Notification Provider" for every app
That is too much I think? It would be weird to trust one notification provider for some stuff, and another one for others? Every user should be able to choose a "Notification Provider" is IMO enough.
On that front, there is already a push standard which is called WebPush which allows to do this seemlessly. Except it's only in browsers (also I don't think any major browser allows customizing it?). On non-Google Android, you have UnifiedPush. (It isn't a standard. Not sure it would make sense to make a standard of an Android API?)
I've been thinking of making a OwnerOS: a bare Android who lets users pick every component they want. You select which store you want, which assistant you want, which notification provider you want, which SystemUI you want etc. All of which currently require to root your device to customize.
> On that front, there is already a push standard which is called WebPush which allows to do this seemlessly. Except it's only in browsers
My understanding is that this uses centralized servers maintained by each browser vendor.
This post is much more informative then I'd expected from title and comments. It opens with:
> We're here to stop surveillance by corporations like Google and Apple. That's why we replaced Google’s FCM with our own notification system and keep Apple Notification Data at a minimum. Read on to learn why this is important.
Edit/Note to self (and maybe others): Be conscious of how much time I spend reading about actually doing things and their details vs reading/discussing a meta to actually doing things. Certain meta discussions are useful, others only feel useful but don't amount to anything.
Why not just use UnifiedPush, which is already an open standard? Or, like other libre apps are doing, provide an option for external open providers, such as ntfy, Nextcloud Push etc.?
I agree with the writer, SSEs have a lot of untapped potential and they're way less resource-hungry than Websockets. But if every app implements its own SSE manager, it'll still be a lot of overhead on the system as a whole. Better rely on a 3rd+party open app like ntfy that takes care of forwarding/dispatching all the notifications, and other apps don't need to create a separate listener.
Link for UP https://unifiedpush.org
People really don't understand the attack here.
It does not matter who is providing the notification service. No amount of encryption (actual E2EE encryption) prevents that ability for a government agency or criminal enterprise to functionally compromise the service to determine which users are getting push notifications from which other users or services.
It also does not matter if you use push notifications (which are vastly better for performance by every metric), or polling. Necessarily the intermediary (Apple, Google, Signal, FB, etc) know the origin and the destinations of anything that would currently be a notification. Requiring polling does not stop that.
Having lots of different services does not stop it either: the orders given to google and apple can just as easily be given to any other company or organization, and more importantly it sounds like google and apple were only able to say anything because a US Senator explicitly asked them so we have no way to know if any organization that was not explicitly asked is also subject to the same orders. The same applies to a criminal organization compromising such a service, only providers aren't prohibited from saying anything, they're just oblivious.
If you are using a service that necessarily involves a third party, that third party can be subject to orders that require them to turn over anything about you or messages you send or receive, or criminals compromising the provider watching the same thing. Encryption (real encryption, not just TLS, not "no one other than you or the provider can access it") can only protect the actual content, the sender and the receiver cannot be protected.
There are systems like MIT's Alpenhorn and Vuvuzela that protect metadata like push notifications by using encryption and deliberately adding noise to foil traffic analysis. Notably both sender and receiver are kept private and you do not need an out-of-band key exchange mechanism to initiate communications for the first time.
I've wondered if signal would be more secure if each signal account periodically sent a message to another signal message. The client would of course decrypt, notice it's just a fake/noise message, reply to it, and then delete it.
Somebody, somewhere has map the fact that user ABC wants their push notifications delivered to device XYZ. That somebody will always respond to legal requests demanding information about this mapping, and keep it secret if legally required.
Nobody is going to break the law on your behalf. Nobody. Not even this smug email provider who did what they did because they didn't want Google to have metadata.
Developer documentation has stated, from the very beginning, not to put sensitive info into push notifications. If you absolutely must, encrypt it with a key that they don't have. An ideal push notification is "Hi", and the app should know what to do with that. Whatever shows up on your phone screen was generated entirely on your phone and isn't sent to any server, and can't be recovered using these legal requests. Unless the app developer is stupid, in which case why would you think that another service is going to change that fact?
We already have a Push API specification that supports arbitrary push server URLs and self-hosting. It's called the Web Push API, although it can be used for mobile push as well.
The only problem is that no one is using it.
A bit longer comment on it:
https://gist.github.com/indigane/70ed13d5287c2d18b3e8e5d4f0c...
First of all, that's a browser API while the article is talking about an Android app.
Second, the Web Push API is just a unified API for websites to use Google/Apple/Mozilla notification services, which the article is trying to avoid.
The last thing i want is several "notification providers" feeding me notification spam
Fingers crossed, a well promoted campaign in EU may force apples hand… again..
Why don't people disable notifications? It must be really annoying if you have 10 apps and every hour they distract you with messages like "buy now! A new product with 10% discount (and 300% markup)" or "where are you? you haven't been watching videos from StupidBlogger for 1 hour".
Any app that does that gets removed.
I ditched a samsung galaxy note because of that, it would do stupid things like send me a notification for each local NFL game, which I couldn't disable. Amazon's kindle app and amazon prime video app did similar.
It's pretty easy, I'd expect anyone that values their attention to remove the app or silence the notifications.
I do have many notifications disabled, but there are still plenty I find useful. I allow messaging apps to both put up a push notification, and vibrate my phone. My email app (and others) can put up a notification but isn't allowed to bother me with a vibration.
I don't get why people allow marketing notifications. If an app gives me a marketing notification without a way to disable it, that app gets uninstalled immediately. (Apps shouldn't even put up marketing notifications without you opting in first, but sadly this is the world we live in.)
Push notifications are incredibly useful, especially for as private a thing as messaging. “Just disable notifications” isn’t a real solution for almost anyone.
Apps that do that don’t last long in my phone.
But sometimes the notifications are useful… taxi/Uber has arrived, SMS/Messages, calendar reminders, important Slack channels, etc.
Youtube does that as far as I am aware.
No clue, I use the website.
One obvious alternative: eschew push for as-high-as-practical latency automated pull.
Pull does not solve the problem. The attack here is the metadata "what was the origin" and "what is the destination", and pulling data does not solve that.
There was some discussion on techniques to get around that in this HN post about a messenger type product: https://news.ycombinator.com/item?id=37105477
Sorry, I thought the problem was that the pushes were going through Apple's and Google's systems, which direct pulling would avoid.
How about messaging via wide-spectrum dead-drop hopping? (redundant array of inexpensive drops?)
Why not just an SMTP extension. Send notifications via email, which expire after X seconds. No need to reinvent the mail
Because such a protocol has to be designed with battery efficiency and mobile data usage in mind. Typical message should be no more than 1 byte.
I like Pushover's protocol design: <https://pushover.net/api/client#websocket>
The interesting part is not in sending the messages but in receiving them. How are you going to fetch that email?
IMAP IDLE?
Yes. And then as with that solution, in many cases it doesn’t work and you have to fall back to regular polling. If you remember to detect that the connection has dropped. And then the testing defeats the purpose because it wakes up the radios.
So now you’ve packaged the problem in a new layer that doesn’t solve anything but creates new problems.
The concept of an open connection waiting for the server to say anything is not novel or special, it just, as we already know, doesn’t really work reliably, you have to do more work to make it reliable.
The pushing services must be running in the background constantly to provide the push notification services. This is how apps can receive notifications without even running it.
If every app uses its own push notification services, an average consumer may end up with half dozens of those notification agents running consuming more RAM and battery life. For Android devices this might be OK (8-16GB RAM nowadays) but for Apple, this may not even be possible for its RAM size (<=8GB).
Just remove notifications entirely, they're just noise.
Remove cell phone completely, they’re just noise.
Remove computers completely, they’re just noise.
Nothing but Stone Age tech allowed.
stone age tech is to noisy. didn't you watch the flintstones? ;-)
rich coming from tuta who still lack a onion based login. this ticket from 2018 was locked as off-topic. https://github.com/tutao/tutanota/issues/528
as lenin said, the best way to control the opposition is to lead it. for me, unless the company has been raided by the government they simply cannot be trusted.
apple proudly advertises privacy on huge billboards while sharing everything they are asked under shadow laws. absolute hypocrisy and double standards. but then they wouldnt be where they are without government money and favours.
It’s hypocrisy and double standards to follow the law? You say “Shadow law” as though it’s them being nefarious.
I don’t know anyone who interprets Apple’s stance on Privacy as “They will break the law for you”, only in contrast to… everyone else (?) selling your advertising data?
It is hypocrisy to advertise yourself as something that you are not. Yours is a deliberate mis-telling or misinterpretation to make them appear saying something they're not.
> I don’t know anyone who interprets Apple’s stance on Privacy as “They will break the law for you”, only in contrast to… everyone else (?) selling your advertising data?
I know that most people will assume that everything is private and hidden away based on their marketing campaigns; assuming that privacy = breaking the law is something you are foisting on, and is in poor form.
Either you are deep inside the HN/Apple reality distortion field or avoiding acknowledgment that a problem has existed for a long time, which needs addressing. This isn't uncommon on HN sadly, I will usually see similar mental gymnastic performed whenever there's any slightly negative news about Apple.
> assuming that privacy = breaking the law is something you are foisting on, and is in poor form.
What privacy do you have that isn't subject to the law? Outside of your own brain, can't literally everything be subject to a legal order to gain access?
It seems to me that your definition of absolute privacy is extreme, extremely specific to you, and renders the word essentially meaningless; This does of course make it very easy to argue in bad faith and rant about companies that you don't like.
I think the GP's point is that Apple's marketing may make it seem (to many people) like Apple's security is set up in such a way that the law isn't relevant, and that users' data is encrypted and handled in such a way that even if Apple is ordered by a court to divulge something, they are simply unable to do so.
And that's true, to some limited extent, but (some) people believe it's true to an absolute extent.
Apple doesn't need to tell people they will break the law in order to maintain their privacy; they just need to insinuate that user data is handled in a way such that it's out of the law's reach.
> Either you are deep inside the HN/Apple reality distortion field or avoiding acknowledgment that a problem has existed for a long time, which needs addressing. This isn't uncommon on HN sadly, I will usually see similar mental gymnastic performed whenever there's any slightly negative news about Apple.
Or, it could be that other people have legitimate views and interpretations that don't match your own?
By your own logic, you are also doing similar mental gymnastics to interpret news about what a government is doing in a deliberately negative way about Apple, driven by an explicit marketing piece for a company that is trying to present themselves as the solution.
I really want to like onion, but literally the only thing I can reliably find is search engines with drug and porn ads, and most results to any query I do throw in are non-responsive, and those that aren't broken are still incredibly slow.
Well, let's put an end to that literally:
Facebook is a cesspit on the normal web, is it really better on tor/onion?