alcopop.org

Introduction

I used the default MTA in Debian, exim, in conjunction with the 'mutt' mailreader for a long time without looking at the way I used email and how it could be improved. I had many different email accounts, used for different purposes. I used the same machine for a mail server and for where I read and stored my mail. I also used 'procmail' to do some very basic filtering and sorting.

Mail format

The default mailbox type which exim and mutt were using was 'mbox' format. This stores each mail in a given box in a single file. Each email is delimited with a special _FROM line, and occurences of this magic string in the headers or body of email need to be escaped.

Some of my mailboxes involved big attachments (logs etc.) and when they reached 40MB in size, it took an incredible amount of time to open the mailbox (as the entire file had to be scanned) or make any changes.

I've sinced moved over to Maildir format, which used a directory for the mail box and a set of files for each email. In general this has increased the speed of opening and changing mailboxes significantly for the better.

Having said that, Maildir suffers when instead of having large quantities of mail in bytes, you have lots of individual mails. There's a mature patch for mutt which caches the headers which increases speed at a cost of disk-space (which doesn't matter much really).

Maildir also saves messages using the Message-Id field as the filename, meaning you can't copy the files onto windows partitions.

Exim

I'm not sure how to make exim use Maildirs, but I used a procmail recipe to automatically empty my mail-spool file (more on this later).

Mutt

This simple directive changed the default mailbox format from mbox to Maildir for mutt:

set mbox_type=Maildir

Procmail

Procmail was more subtle but still relatively straightforward: all references to mailboxes needed an appended '/'.

Fetchmail

I think fetchmail is well-behaved and delegates mailbox type responsibility to the MTA, so no configuration is necessary.

Sent mail

When you receive mail, you sort it into folders and possibly into a heirarchy (if you are using something better than mbox or Maildir, at least). Thus, when it comes to finding that information at a later date, you can jump to it with relative ease.

(Of course, google's approach is that even this isn't good enough and a hyper-intelligent search should find what you are looking for.)

But, we typically don't treat mail we send with the same respect. Most of the time we don't even keep it - sometimes though, people save it in a folder (say =sent, or 'Sent Items').

Via the mutt FAQ, you can put a copy of your sent items into the current folder if you have the current_shortcut patch applied. This makes '^' a mnemonic for the current folder. You then just make sure that the rule is re-evaluated whenever your folder changes:

 folder-hook . 'set record="^"'

/var/mail/jon vs. ~/mbox

Exim stuck new mail in the mbox file /var/mail/jon, whilst mutt would ask me if I wanted to move read messages to ~/mbox. I was treating both locations in the same way: a place to leave mail which hadn't been dealt with. Had it been dealt with, I should move it to another mail box or delete it.

It was annoying to have a directory called ~/Mail for my mailboxes, with the exception of ~/mbox. I instructed mutt to use ~/Mail/inbox/ in place of where it would use ~/mbox. I moved from mbox to inbox, since I am not using mbox format anymore (so we might as well purge the use of the term):

set mbox=~/Mail/inbox/

We need to watch this box for mail too:

mailboxes ... =inbox

I added a catch-all rule to procmail which moved all mail from /var/mail/jon straight to my inbox. Now I only have to manage my inbox:

:0
$MAILDIR/inbox/

Finally, I want mutt to start in =inbox rather than /var/mail/jon which should be empty:

$spoolfile = ~/Mail/inbox

Mailing lists

Most people who are new to mailing lists and use mutt hit 'r' to reply to a post and end up mailing the poster directly. For this reason, I added this alias so I don't:

        macro index r <list-reply><reply>
        macro pager r <list-reply><reply>

Thanks to Florian Ernst for this tip :-)

Note: this doesn't work in woody's mutt (Mutt 1.3.28i (2002-03-13)) or Mutt 1.4.2.1i (2004-02-12). It used to work for me, though. Maybe it relies on side-effect behaviour or a patch in Debian/sid's mutt.

Directory Queries

My -exUniversity has an LDAP server for mail clients (and other things, I suppose) to query. I managed to get mutt to use it, with the help of this perl wrapper script, suitably modified.

I had to specify an older LDAP protocol (-P 2) in order for it to work. Once working, I can do queries using Q in index-mode, or when prompted for an address, I can do completion using ^L.

Where next?

The procmail format is just horrible, although procmail is simple and effective - I'd like to see if there is an alternative which is easier to use. Also, either way I'd like something which could verify/validate my recipes, as if I get one wrong the first I hear is when someone complains about bounces or similar.

One disadvantage of using a seperate MTA, MUA, filter, fetcher, etc. is that if I want to move from say, ~/Mail/inbox to another name (like I did from mbox to inbox); I need to change more than one config file, and pay particular attention to the order I change them in. For example, if I changed the procmail recipe to automatically deliver to a new Maildir which didn't yet exist, lord knows what would happen - so I have to use mutt to save all of the existing folder to the new one (and create it); then change the procmail recipe; then change the mutt config; then finally deal with any email which arrived in the middle of my changes.

I'd like to either have a user-specific MTA or know better how to handle my existing one. I'm from the netscape/outlook school of email originally, although I love mutt and I believe in the unix philosophy of 'do one job well'. So I need to learn how to handle multiple email accounts (of various types) and how to send some emails from one, others from another, etc.

Fetchmail constantly goes wrong when I receive large emails at a particular server. I would file a bug report on this, but I'd have to leave my mail server stagnating whilst it went on. Additionally, it is *very* annoying to wait for a slow mail server to respond when another, lightening fast one could be being processed at the same time, yet esr has resisted any attempt to have concurrent support added to it. Basically fetchmail works for what I use it for, but it seems very limited and a bit easy to break so I'd like to look for something better.

Several years later, and I think fetchmail really sucks. I use retchmail these days.

Finally, there exist patches which add NNTP support to mutt. I love mutt as a mail-reader, and I think it would be very useful as a news-reader too, but incorporating nntp support is a bad idea. I'd like to investigate the possibility of a local mail-to-news gateway which was transparent to mutt, and transparent to the groups at the other end, too.