Show HN: Mailready – meet the new email deliverability standards
mailready.infoSome of these popped up on Hacker New recently;
- Learn and Test DMARC[1] does a visual breakdown of how email servers communicate, giving you a better understanding of SPF, DKIM, and DMARC and how they work together.
- Mail-Tester[2] - test the spammyness of your emails.
- MECSA[3] is an online tool developed by the Joint Research Centre (JRC)[4] to assess the security of email communication between providers.
2. https://www.mail-tester.com
this doesn't seem to help for private email.
i want to send email to my friends at google. yet google blocks delivery.
this is not any kind of business or commercial messages. but from my private account to my friends account.
SPF and DMARC check out and surely private emails should not need unsubscribe headers. so your site says everything is fine. then why does google still reject my emails?
This really getting crazy. My daughter nearly did not get into the swimming course because google just black holes the registration confirmation because my wife's used her Gmail.
I really hope that this kind of stuff gets illegal: just taking an email and virtually burning it.
this really sums up my feelings. i have sent emails to people past that i have no other way to reach and never got a reply, and i have no idea if they even got my email.
and the same the other way around. which is one reason why i run my own server.
i always believed that spam filtering must be done at the end user, and noone else has the right to block email from reaching me. in particular the most obvious thing, every address that i send to, should automatically be whitelisted as a valid sender, unless i explicitly mark it as spam. the exceptions should be obvious DMARC/DKIM/SPF violations.
at one point i was even working on my own email server to implement this kind of whitelisting/filtering myself.
My social group has been shifting away from using the internet-wide email system to using a private one just among us that we run. It works well in my group because most of the emails we send/receive are amongst ourselves anyway.
All of these antispam measures are fighting a losing battle -- every one of them reduces the utility of email and are only (barely) acceptable because spammers reduce the utility of email to an even greater degree.
By running our own email system that doesn't interconnect with the internet's, email has become actually useful again.
Hah... somehow i have this feeling that we are heading right back into the "good old days" of separated BBS networks, some commercial, some private, but none of them interconnected.
Somehow i like the thought of this...
We've already done this. In effect, we're running a private "internet" that uses the public internet as one of the communications channels, but does not interact with any internet servers beyond that.
It's really beautiful and freeing to have an "internet" that works really well, even if it is a very tiny one.
But running an email system that does not connect to the wider internet also ruins the idea of email as a globally reachable address. Everything is a trade-off :-|
True, but I think the days of email as a globally reachable address are already on their way out. It's not exactly unheard of for some people to be unable to send email to addresses hosted by some common mailservers (including gmail) right now.
However, none of my friends use our mailservers exclusively. They also use the internet mail system. But having our own means that we don't have to worry about deliverability issues, spam, or any of the other problems that exist on the public system.
IP-based rejection may be the answer. You may not be doing anything problematic but if your IP neighbors misbehave, your IP will be blacklisted too.
i am considering that, but it has worked before, and i have this same IP for years now. (i am not 100% certain, but i am pretty sure i already had this same IP when it did work)
anyways, my suggestion here would be that an IP check would be a feature that mailready.info could include.
> then why does google still reject my emails?
Multiple options. For example, your IP address may not have a good reputation. This can happen when a previous tennant used your IP address to send spam, but it also happens when you send very little email to Google/Microsoft servers, not giving you the opportunity to build a good reputation. I briefly considered sending my mail server logs to Gmail so I could get regular whitelisted email delivered, but I changed my mind when I realised Google would probably mark my domain as a bot.
This seems particularly bad on IPv6 for some reason. I'm not sure why, maybe it's because their spam filters are treating every address as a /128 rather than a /64 network?
The worst server in my experience is Microsoft Exchange. I caught the stupid platform taking my email, _rewriting the email address because it didn't like it (despite being compliant!)_, and _then_ checking the DKIM signature, which obviously failed. It doesn't have IPv6 deliverability issues, though, because like many Microsoft cloud products, it doesn't even support IPv6. Microsoft Outlook also sometimes fails the SPF check... because of DNS issues _on Microsoft's side_.
None of this is standards compliant, of course. The best you can do is DKIM+SPF+reverse PTR+strict DMARC+DNSSEC+DANE+using some expensive data center so there aren't many spammers in the nearby IPv4 blocks. Most of these can be generated automatically through online tools or ready-out-of-the-box email servers such as Mailinabox or Mailcow.
Also, _check your configuration regularly_, set up alerts or something; sometimes something may break and your domain/email address will start losing reputation.
It's infuriating to get email delivered, even if you do everything right. I've given up on that stuff, though, and tell everyone I email to check their spam folder and move it to their inbox to train their spam filter.
when you send very little email to Google/Microsoft servers, not giving you the opportunity to build a good reputation
this is something i find really frustrating, because, how am i supposed to fix that?
it's a personal server. there simply isn't that much outgoing traffic. and then, because google rejects my emails i have to use a different server to send mails to gmail.
so how exactly would i generate that neessary traffic that unblocks me? (this is kind of a rethorical question, i don't expect a real answer here because i don't believe a real answer exists)
should i write every email twice? from two different senders? i feel that would make the emails even more suspect than making things better.
send fake emails? that would be like sending spam in order to convince google that i am not sending spam.
seems to me that if low traffic is really the reason then there is no hope, and all i can do is to give up, which for now is what i did.
Exchange is an abomination. A few hours trying to do something with groups was enough to see that for me. As a QA I would reject it until 100 or so bugs were fixed (that I saw in a few hours of use, so maybe 10k+ bugs total?)
> What happens if I dont comply? >You get marked as spam? And that's probably the best case scenario. Your email may not get delivered.
My best friend works at a large ISP specifically on their email transport system. They discard 97% of the emails they receive. That's straight into the bit bucket, not to your Junk Mail folder.
hotmail or comcast? My users complain about email just disappearing, no bounce, no spam folders, just gone.
I can’t disclose his place of employment. He did say this was kinda the reality the big email domains contend with.
I have seen something similar at a smaller ISP, but the silently discarded email usually came from domains with bad SPF setups.
Really infuriating, because customers would not believe that this wasn't a problem on our end, it was the other side telling us to discard their email!
You wanna know what's worse? There is an SPAM Scoring Monoculture, dominated by BrigthWorks, Symantec and another firm that escapes my mind right now. So if users flag enough of your email as Junk on an ESP using one of these providers, your domain is banned or auto-Junked with all that company's other customers.
SPF alone is not a reason for discarding a letter.
It shouldn't be, but the standard is not clear.
The spec is very dodgy in that it acknowledges that every server needs to pick its own policies:
> Disposition of SPF fail messages is a matter of local policy.
On silently dropping email, the following is listed as a "consideration":
> Other dispositions such as "dropping" or deleting email after acceptance are inappropriate because they leave uncertainty and reduce the overall reliability and utility of email across the Internet.
There is no MUST (NOT) in the spec when it comes to silently dropping email.
The intent of the author of the spec is to always provide feedback, but it doesn't actually say that in cleae terms.
"Send reconfirmation emails to people that have not interacted with your email (no opens or clicks)"
How can I check opens? I'm not aware of any reliable way to check this. Mail clients not loading pixels means the software is unaware of opens?
I personally tend to enable delivery/read receipts for new email domains. Big providers will often send you a notification for delivery at least, and prompt for the read receipt.
You can also track your DMARC statistics and figure out what mail domains tend to not deliver your email.
Same way as always, image loading. If image URLs are unique and external then you can track their opening. Not all users will allow it so you also track clicking on in email links.
Doesn't just about every mail service proxy, and often preemptively fetch those?
Not every service, buy many of them do.
I'm quite sure GMail does... which is basically the entire internet as far as mail delivery.
Maybe because they click a link in the confirmation email?
Our domain checks out because we use O365. However, we have an old Exchange server sending out via SMTP. We're not sure what the best path forward is for us. Do we change our apps to route through O365? Will probably take days for that. We have in house custom apps that use it heavily. Anyone have good resources on what we should do?
With the appropriate configuration, your SMTP server could relay to O365, just acting as a forwarder, you don't necessarily have to remove the server. This is very common in use-cases for old devices that barely support SMTP authentication, never mind TLS!
There's an oss dikim signing transport plugin for on prem exchange that has become somewhat a standard in this space.
Thanks for this. Will fix.
Your fix is wrong. It states the SPF record is invalid. It is not.
https://datatracker.ietf.org/doc/html/rfc7208
> Unrecognized modifiers MUST be ignored no matter where, or how often, they appear in a record. This allows implementations conforming to this document to gracefully handle records with modifiers that are defined in other specifications.
A correct SPF validator will ignore the xss modifier, not treat the SPF record as invalid.
If you have SPF but not DKIM is that sufficient for deliverability to these providers?
For DMARC alignment you need either SPF alignment or DKIM alignment, either one will do. (note that 'alignment' is not the same as an SPF pass).
Which means that you _could_ get away with just SPF alignment, but you wouldn't want to trust on that since SPF is horribly broken and most third party senders don't even bother with SPF alignment anymore. Always focus on DKIM alignment instead.
But if you are now just thinking about this, you're in trouble anyway. If you are sending bulk amounts of email (that is, 5k a day per Google's rules) and you are not yet signing with DKIM, then you are probably not ready for adopting a strong DMARC policy ('quarantine' or 'reject') before Feb 1st.
Email hardening takes time, the larger/more complex your domain is, the more time you probably need to ensure you are DKIM aligned for all your delegated senders. Don't be tempted to just add a DMARC record with p=reject policy, that would be irresponsible and asking for problems (read: undeliverable email).
For mail that you're sending yourself, from your own infrastructure (e.g. the envelope-from address matches the header-from address), DMARC + SPF should be sufficient and is easy to implement.
For third parties that are sending on your behalf, you'll likely need DKIM - but that will be implemented on their side, and all you'll have to do is add the DNS record they give you.