Jeremy Johnstone

Ok, so I have a problem. One which keeps me up at night and annoys me greatly, but I can’t seem to find a solution to fix. So what is this problem? Email SUCKS, that’s the problem! Ok Ok, I am sure you are thinking “great, and?” as it’s a common rant. Here’s my needs:

1. I want my email everywhere at every time, with or without network access, on a range of devices. Sitting at my desk with my main desktop? Sitting in a meeting with my laptop? Sipping a coffee with my iPhone in my hand? On a plane flight to some far away place? All of those should be no challenge to doing something as simple as reading an email, right?
2. I like to organize my email, especially mailing lists which I subscribe to hundreds of at last count. Some lists are important, others less so and should be organized accordingly. As having things “in view” can be distracting when I don’t need to see them, I need the ability to have a nesting system or some other way to have multiple views into my mail. These “views” should be synchronized across all devices and possibly have the ability to be device specific.
3. When someone sends me an email, it should show up instantly in every client, not five minutes later, not an hour later, certainly not forcing me to restart the app just to see the email. The system should also support some type of push system as constantly “polling” puts excessive load on the server and sucks up precious power reserves in the case of mobile devices.
4. I need the ability to find old stuff quickly and painlessly, again on any device. We can have Yahoo! search teh net in under 100ms for anything I could dream up for a search term but even my quad core Xeon can’t accomplish a similar search across an infinitely smaller dataset in under five minutes? Huh? You’re kidding right? Guess I won’t even try it on my iPhone then as I know my battery will then run dead before the search finishes in that case.

Seems like fairly modest needs right? Should be doable? Well, I am amazed to say it seems to be about as simple as sending a man in a balloon to the moon.

Currently I am running my own Courier IMAP server, using Procmail and all it’s powerful mojo to sort my mail, and using Mail.app as the actual MUA. The problems I run into are as follows, in no special order and certainly not all inclusive:

1. My mail server is constantly thrashing the disk. Despite being on a dual G5 w/ plenty of memory and decently fast I/O, it seems the combination of commands Mail.app sends to Courier seems to give it fits. Have two clients hitting it at once and the machine drops to a snail crawl. Heaven forbid you add that third client and the machine starts sparking and flames shoot out the back. Of course running it on another OS would probably help as OSX doesn’t have the best kernel support for things like this, but that’s not the point. I’ve also thought about moving over to another IMAP server implementation, but as yet I haven’t found one which I feel confident will be better performing without enough negative aspects to scare me away.
2. Mail.app seems to not like to pull subfolders with consistency. It’s extremely common for me to restart the app and suddenly have hundreds of unread emails suddenly appear across folders. This definitely is a problem no matter how you look at it. Of course there are multiple possible causes to blame, but that’s irrelevant. I don’t care if the IMAP spec is brain dead in how it specifies IDLE support, that’s not my problem. Having a mail client which shows new mail without fail should be one of it’s developer’s top priorities.
3. Procmail while very powerful is extremely error prone. I’ve had several occasions where things backfired. My dream would be to someday have something which said “hey, you moved this email into this place, would you like me to move emails like it in the future for you?”, but then I snap back to reality and realize that’s a geek’s wet dream and not anything which will really work as you would expect any time soon.
4. Keeping multiple devices and offline copies of emails in sync seems to be error prone at best. Isn’t this exactly the problem IMAP is supposed to solve? Tell me again why there is not a single IMAP client which handles this painlessly? Oh, right, that’s because the IMAP spec is crap. Please fix, ok? KTHXBAI!

So my question is, is there something I have missed? If you have a solution I am more than ready to apply my hand to my forehead and yell “Do’t” with enthusiasm. Sound off in the comments with what works for you.

Tags: courier, email, imap, iPhone, mail.app, procmail, rant