Ask HN: Is the state of mail user agents that sad?
A lot has been said about how and why e-mail is broken (such as [1, 2]) and how encrypted e-mail is a lost cause (such as [3, 4]). This is all true and sad enough. My gripe (both current and perennial), however, pertains to Mail User Agents (MUAs).
First off, we have to state that many people don't even use a MUA at all, but instead rely on GMail and other webmail, which runs in the browser, and the problems with modern browsers are beyond the scope of this thread; suffice it to say that I deem `modern web applications' too bloated, much like Drew Devault and others (see, i.a., [5]). (Just to be clear, this does rule out Electron-based MUAs for me.)
I used to use Thunderbird for almost forever, until I found that it had inexplicably high CPU usage every now and then (not caused by the search index, mind you), and it was a bit tedious to configure for plain-text mail [6]. Besides, Mozilla has threatened to drop this project more than once. So I tried a few other clients (such as Geary and Evolution) and switched to Claws mail.
Claws mail does have outstanding performance. But its source code was not very convincing (mixing several layers of abstraction, including GUI, in the same place). Besides, it's a GTK program.
I just can't help it — I find GTK very hard to configure: * On a fresh install, it is very well possible that some icons or mouse cursors are missing. * On some systems, I have to configure using the ini file. On others, it is some arcane gsettings thing. * You need to play around with themes for it to look decent, so good luck with the gsettings, the missing icons, and what-not once again.
Speaking of plain-text mail (as I did a few paragraphs ago), the website I mentioned [6] does list quite a few clients with textual user interface or command-line interface. I tried aerc, because it was fresh, written in Go, and because I do respect the principal author, above-mentioned Drew Devault, very much. But I found that the UX was not to my liking, and I don't quite understand why a MUA should include a terminal emulator. Or why it should be a self-contained application altogether.
This whole idea of a self-contained MUA application goes against the general UNIX-y paradigm of "no application" [7] and "choose extend over embed" [8]. Don't app me in, if I may say so.
Other MUAs, such as nhm/mmh or neatmail do in fact follow these paradigms. But their adoption seems very weak, and: * they are written in C (which is error-prone and also not the best language for quickly testing hypotheses regarding MUA design), * neatmail uses mbox, which I find rather limiting, * nmh/mmh do use Maildir, but they depend on context information (such as `current message') that they have to manage in the file system, instead of expecting the necessary information as parameters to the operations.
Now, in order to use these UNIX-y programs, I imagine (though I didn't check) one also needs dedicated programs for communicating with mail servers, i.e., for receiving and sending mail. So I read up a bit on sendmail, postfix, qmail, fetchmail, getmail, and fdm. And we seem to be treading buffer-overflow territory once again (or deprecated Python 2 in the case of getmail). I find this rather sad.
Am I missing something? Is the state of MUAs and e-mail in general really this bad/sad?
[1] https://doctorow.scribe.rip/dead-letters-73924aa19f9d
[2] https://ploum.net/the-monstrosity-email-has-become/
[3] https://latacora.micro.blog/2020/02/19/stop-using-encrypted.html
[4] https://words.filippo.io/giving-up-on-long-term-pgp/
[5] https://drewdevault.com/2020/08/13/Web-browsers-need-to-stop.html
[6] https://useplaintext.email/
[7] http://wiki.c2.com/?NoApplication
[8] http://wiki.c2.com/?EmbedVsExtend I'm still fine with Thunderbird as a MUA (more or less). Using number 1 and 4 as shortcuts to mark an email as "todo" or "urgent". Invitation to appointments (ics file format) can be accepted as well. Integration of contacts is a paint point though. For email automation I have written a lightweight CLI in Rust [1] which supports MIME and SMTP. I'm currently integrating sendmail's capabilities as well, so that it can be used both as a MUA (message user agent) and MTA (message transfer agent). Cargo's feature flags will hopefully give you a UNIX-like experience (only compile the features you want to use). In general, I share your sentiment. Email is still the best protocol to contact people outside of your organization. It's a pitty that encrypted email somehow haven't prevailed. The matrix protocol might be better suited for standardized and encrypted communication. I would argue that every citizen should get a state-issued email address which then can be used for official communication with public authorities (signing and encrypting being a prerequisite). In Germany, you still get so much paper via regular mail. Even the ten-page Covid-19 quarantine order is send by post. It only arrives when quarantine is already over. (Admittedly, you get this order by phone, too, and they are asking for your email address.) Thank you for responding, and for providing me with a link to your project. It is good to see that things like this exist in languages other than C. While some people on this thread in fact implied my constraints were unreasonable, I have to say I'm a bit fed up with all the statements on Wikipedia regarding security vulnerabilities of common mail software -- due to buffer overflows. It seems to me that majority of users don't want to deal with a native application - specially one that: 1. They would have to install, maintain, backup and configure themselves
2. Has a reasonable web application, that, as a major perk, they don't have to install, maintain, backup and configure themselves Due to the lack of demand for such a product, I am not surprised the existing products arn't polished/well maintained. Yes, I totally see your point. Gmail and similar offers are convenient, which means that native software has some very stiff competition and not a lot of adoption. I'm interested in frugal computing, and I want my software to run smoothly on a Raspberry Pi if at all possible. So I do have some demand for a native program or, rather, set of programs. Besides, on a principle level, I fail to see how something as simple as e-mail should need (or be allowed to use) something as complicated and resource-hungry as a web browser. > I'm interested in frugal computing, and I want my software to run smoothly on a Raspberry Pi if at all possible. So I do have some demand for a native program or, rather, set of programs. I understand. My point is that it's likely you're a fraction of the overall "home PC" market who are fine purchasing a regular PC/Laptop/Tablet with 16GB+ RAM and multiple cores (because it's so cheap now) if that can support apps that simplify their overhead. SO while a Raspberry Pi with a NAS that backsup to rsync.net might consume way less power and be more cost efficient, it might require technical overhead that majority of people are unwilling/unable to support/incur. > Besides, on a principle level, I fail to see how something as simple as e-mail should need (or be allowed to use) something as complicated and resource-hungry as a web browser From a developer of a product POV, it's much simpler and less of an overhead building a web app that runs in a web browser than a native app that runs on 5 different platforms and architectures. Essentially the developer of products are outsourcing the headaches of 5 different platform and architecture compatibility to the web browser. Again, I have to agree with you. I know I don't represent the majority, and I know that web apps have advantages (at least in this economy, so long as natural resources are cheaper than programmer hours). I don't argue with that. But I still find it a bit sad ;) Time for a shameless plug. I'm working on a decentralized, end-to-end encrypted email, and it is currently functioning, with support for POP3/IMAP clients, as well as a web client. The idea is keep the identity registry on a blockchain, so your identity cannot be taken away from you. Messages between users are always encrypted, in transit or at rest. We also have a gateway that allows you to use the system as a regular email, and proxy server that translates our internal API (protobufs, gRPC) to the legacy email protocols, SMTP/POP3/IMAP. Try notmuch I used the text-based Pine (now Alpine) on Windows as my MUA a while back, and enjoyed it quite a bit. It let me use Vim as my editor and Par as a powerful text formatter. Those, along with its speed and keyboard friendliness made working in a Windows environment bearable for this old Xenix hand. And no, I didn't have to set up any mail servers to make it work, I just used existing SMTP and IMAP servers at work. >This whole idea of a self-contained MUA application goes against the general UNIX-y paradigm I don't understand this at all, or your concerns about buffer overflows and "quickly testing hypotheses regarding MUA design." Text-based MUAs in particular are very solid and fast; find one you like and quit worrying about irrelevant issues. Or, learn Emacs and use one of the mail clients it supports (or write your own!). —signed, former tech support for Z-Mail and text-based Z-Mail Lite I have been using mutt for triage and simple things, with Google web mail interface when necessary for e.g. HTML mail. The workflow is ok. I get a large amount of email that I can just delete, and it's particularly nice to be able to select a bunch of cron emails with a regular expression. In the last year it's become unreliable. When I move a message from one folder to another, it doesn't stay moved. I guess it's some error handling problem. I have lots of emails in some folders (10k). I have thought about writing a driver for mutt that works directly against the Google API instead of IMAP, but writing REST code in C would be a slog. Original author here. I have the following desiderata concerning MUA: * it should be able to work offline (with a dedicated send/receive cycle), * it should run fine on a Raspberry Pi (out of principle more than anything), * it should favor plaintext mail, * it should integrate well with all kinds of workflows and platforms (such as git email), * it should be a veritable platform for experimentation (i.e., it should be malleable). I think the case can be made that re-inventing the wheel can sometimes be beneficial. And e-mail to me seems rather 'underexposed' (maybe because most people think it has been a solved problem since at least Gmail). > Is the state of MUAs and e-mail in general really this bad/sad? Undeniably, yes. Personally, I still use the last real version of Eudora (7.1.0.9) along with a patch that gives it support for modern security (full TLSv1.2, LE root certs, etc). I use it at home and at work, with over 20 years of email history in each one. I simply cannot find anything that does what it does as well... Handles over 1 million messages with ease Nice filtering rules Super fast search tool Files aren't stored in some inaccessible binary store Keeps attachments stored externally from mailboxes Makes it easy to manage multiple addresses & accounts Very plain-text friendly Has limited image/HTML support & doesn't run javascript (security though inability :) ) etc etc etc I really hope that encrypted email is not a lost cause because that would strongly imply that end to end encrypted messaging is a lost cause in general. If we can overcome the problem in any single case we can overcome the problem generally, even for encrypted email. This isn't something we can overcome with a purely technical solution. The basic ideas behind end to end encrypted messaging will eventually have to be made part of our culture. With respect, I do object. End-to-end encrypted messaging does work, and Signal (among others) is proof of that. With e-mail, either (a) you are backwards compatible and sending unencrypted (even by accident) remains a possibility, or (b) you break compatibility, but then it's no longer e-mail. (Signal is an extreme example of the latter: it just uses its own protocol.) Signal is a good example here because someone did a usability study. In a usability study involving Signal[1], 21 out of 28 computer science students failed to establish and maintain a secure end to end encrypted connection. The usability of end to end encrypted messaging is a serious issue. We should not kid ourselves into thinking it is a solved issue. For all practical purposes it is the issue. [1] https://www.ndss-symposium.org/wp-content/uploads/2018/03/09... This is interesting, and it causes me to reevaluate my stance. At least we have to agree on what we mean when we say that "end-to-end encryption works". I think there are `shades' of "working" if you will -- for instance, I know I mostly ignore when the key material changes in a Signal conversation, and this could be used to fool me. But then we have to talk about attack vectors and what we want to be protected from. I think it's mostly large-scale data collection and analysis rather than targeted attacks (like the CIA might do). At any rate, thanks for setting me straight. I will read the paper! I went down a rabbit-hole last year trying out different MUAs and configurations, searching for that perfect setup. I like CLI interfaces and I wanted something where search was not as painful. It was a nice distraction, then I realized that I only spend a few minutes each week reading email. It's just not a significant part of my workflow, either professionally or personally. For me, it's not just about e-mail, but our industry/discipline in general, and the state of e-mail is a good indicator in my opinion, because almost everybody uses it every day, and its implementation complexity is still rather manageable. Besides, I'm interested in frugal computing and would like to reach a point where I can do my day-to-day stuff on a Raspberry Pi or similar device. The state of MUAs isn't bad/sad unless you apply a pile of unreasonable constraints, as you have. Then, yeah, you can't find anything. Loosen up. "Written in C" is an issue for you? And you use Linux? Just give Evolution another try and give it a little time. I find your answer a bit intrusive, to be honest. Let's just stick to the subject matter instead of telling each other how unreasonable or uptight we are, okay? Yes, "written in C" is an issue for me, for two reasons: * I really do want to do rapid prototyping and testing hypotheses regarding MUA UX, and C is just not suitable for this end. * I keep reading about security vulnerabilities in common mail software caused by (among others) buffer overflows. I don't think that MUAs benefit from C the way system software (such as the kernel) does. And compared to MUAs that run in the browser, I think that the bloat introduced by using Go or Python is still justifiable. Fortunately, there already exists a MUA written in Go (namely aerc). I was just like you. Until a few months ago. I have had mu4e set up for about a year, but I didn't feel comfortable in it, often reverting to the web clients instead, which was miserable. Recently I spent (many) hours making it really my own, and _my god_ what an amazing experience! I have all my accounts in the same view (if I want). I have filters for everything. I can say "follow up" with 2 keystrokes, which will open up a calendar, allow me to pick a date, then the email will be automatically archived and I will not see anything about it until that date in my org-agenda. I use org-msg to compose these Outlook-style email for work (except I can easily quote, post syntax highlighted code, even with the real results embeded, etc). I have 2-letter shortcuts to quickly cut down to the sender domain, the email FROM: address, everything from this address sent only to me, a filter for finding everything that looks like this subject.. I even have 2 letter shortcuts to fire up a browser with this Message-ID and open it (dynamic based on account) directly in the webclient (unless you are MS [1] who for some reason uses POST for their searches. I hack around it by adding a precise query to my clipboard). I don't often use this, but it can be handy. This is all synced using isync. I have two systemd unit files. One which does the inbox-related stuff often, and another full sync which runs every hour if I recall correctly. Finally, since this is all locally, search is instant. And since it is in emacs, I use all my normal shortcuts and everything. I have finally been able to achieve inbox zero, something I never have before. It took a lot of effort to get here, but I love it. Something really amazing would have to come along to displace my workflow, and I don't see that happening. notmuch is another option, which has tags support (mu4e does not). That is the only thing I miss in mu4e. However notmuch stores the tags in a local database which is not easily shared, so for now I am super happy with mu4e. And I often capture the email in org-mode anyway, which does support tags. Most of the config for this in on github [2] (I still have some things to commit), if you are interested. Waiting for tags to be stored in X-headers of the email reliably.. [1]: https://answers.microsoft.com/en-us/outlook_com/forum/outlk_... [2]: https://github.com/bergheim/dotfiles/blob/3cb1540f57f391beb2... Can you recommend a good set of docs and tutorials to set up mu4e? I still use emacs and a separate (mutt) MUA and copy and paste
things between, I know it sounds awful... but hard to escape
entrenched habits. What would you suggest as the best steps for a good setup?
Does everything go in the .emacs config? Do you need to write
much e-lisp or it it a tickbox type config? Thanks for any pointers! They are a bit all over the place - I started this years ago so I don't really have anything solid, but searching gives a few results that looks pretty solid ([1] [2]). If you temporarily tried doom [3], I think you could see how it would work for you. You need to install something like isync as well so you have a mailbox index locally. I would start with just one account - multiple accounts is a bit more effort. In general, be prepared to spend some time getting this to work. I wrote a bunch of elisp, but you don't have to write much. For me it was worth it, but this is not a plug-and-play solution unfortunately (nor is notmuch). [1]: https://jherrlin.github.io/posts/emacs-mu4e/ [2]: http://cachestocaches.com/2017/3/complete-guide-email-emacs-... [3]: https://github.com/hlissner/doom-emacs/blob/master/modules/e... Thank you. After I posted I realised my stupidity in not checking
out your linked config file. Having fun with that now. Cheers for
the additional nfo.