Mail Servers

[Derek Martin]

On Sun, Sep 21, 2003 at 05:20:27PM +0000, dsr at tao.merseine.nu wrote:
> Maildir files are smaller than mbox files (because they only have 1
> message each) so you spend less time opening Maildirs.

Your other arguments are good, but this one is false.  The only time
opening maildir mail folders is faster than opening mbox folders is
when the programmer has gone out of his way to poorly write the code
which opens mbox folders.  There are two reasons why opening maildir
folders is inherently slower than opening mbox:

1. Each message is in a seperate file.

There is substantial overhead for opening a file, on virtually all
operating systems.  Furthermore, the sequence for opening a maildir
mail folder goes like this:

  open -> read -> close -> seek -> open ...

Whereas the sequence for opening an mbox folder goes like this:

  open -> read -> read -> read ... close

With maildir, since the messages are in seperate files on the
filesystem, the probability is much higher that you will need to do a
disk seek in between each and every message than it is  with mbox.  If
the filesystem is unfragmented, the probability of seeking with mbox
is extremely low.  Most Unix-like filesystems maintain a relatively
low level of fragmentation even without tools like MS-DOS's defrag.
Between the overhead of open() and the high probability of seeking
between each message, the impact of message-per-file formats is
substantial.

2. Many files in a directory slow down directory operations.

On most filesystems, searching a directory is an O(n) operation.  The
more files you have in a directory, the longer it will take you to
find a given file in the directory.  Large numbers of messages in a
maildir folder will therefore make processing messages in that folder
very, very slow.


When efficient algorithms for processing mbox and maildir are used,
opening a maildir folder is ALWAYS slower than opening an mbox folder,
even when used with filesystems which optimize for the case of many
small files in a directory.  [This assumes no caching, of course.]
The difference is typically 15%-40% slower.  If the mailboxes are
exceptionally large (in number of messages), the difference can be
much higher, depending on many factors, including which filesystem is
in use.

Your users will notice this, and they will probably complain that mail
is slower than it used to be.  Even though most other operations are
faster with maildir, those other operations are also generally either
performed a lot less frequently, or are relatively short operations
anyway.  The time it takes to open mailboxes is what most users notice
most when they are inclined to complain about mail response time, and
with maildir it is unquestionably slower.  On mutt-users, people make
this complaint all the time when switching to maildir from mbox.
Courier's own benchmarks show that opening maildir is slower than
opening mbox.

As mentioned, there are many other advantages to using maildir, so
this may not be a damning factor for you.  It depends on what sort of
considerations are most valuable to you and your users.  And if you
take care to use filesystems which are well-suited to maildir's
behavior, the performance degredation may be nearly imperceptible to
most users (though benchmarking will show it clearly to be slower).

-- 
Derek D. Martin
http://www.pizzashack.org/
GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.
Replying to it will result in undeliverable mail.
Sorry for the inconvenience.  Thank the spammers.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.blu.org/pipermail/discuss/attachments/20030922/0185faf8/attachment.sig>