[Discuss] Migrating hundreds of mbox files to an IMAP server?

[dbarrett at blazemonger.com (Daniel Barrett)]

Thank you to everyone who responded to my question about importing
large numbers of mbox files to an IMAP server. Here's a summary of
what I learned from you.

* Attempt 1. Rich Pieri and Ron suggested using an IMAP mail client
and dragging mail folders from one server to another. This fails
spectacularly with my mail client, Thunderbird. A subset of the
messages move, and then the operation simply stops without any error
message. This is on a Mac. Thunderbird on Linux is even worse: it
tends to crash outright. I might try it with a different client later
if the following solutions don't work.

* Attempt 2. Gregory Galperin suggested that I use movemail to convert
from mbox to maildir format, and offlineimap to upload to the IMAP
server.  This worked pretty well but had some tricky quirks to
overcome:

  -- movemail sets the timestamp on all message files to the present
moment, and offlineimap uses that timestamp when uploading to the IMAP
server, so all the messages on the IMAP server had the wrong
timestamp. I wrote a script to fix the file timestamps before
uploading.

  -- movemail could not handle a variety of mbox files, throwing the
error "could not create mailbox 'mbox:<name>.mbox': Requested item not
found." Usually this has to do with an incomplete "From " header.

  -- offlineimap requires a config file that took several tries (with
Gregory helping) to get right. The trick was to add "readonly = True"
for the local files. Otherwise offlineimap insisted on syncing in both
directions, which was undesirable.

  -- On Ubuntu, offlineimap is installed by snap. This produces all sorts
of headaches, specifically that snap interprets the symbol "~" not
as your home directory but as some crazy path like ~/snap/offlineimap/297.
As a result, offlineimap kept reporting that it couldn't find my mbox
files. (Have I mentioned that I hate snap?)

* Attempt 3. Dan Ritter suggested mb2md for the maildir conversion. It
seems to be more reliable & robust than movemail and does not suffer
from the file timestamp problem. He also suggested mbsync for the
migration to IMAP but I haven't tried that yet.

  -- One quirk is that mb2md assumes that all relative paths are relative
to your home directory, so simple-looking commands like "mb2md -s ./mybox"
fail mysteriously (because it's looking for ~/./mybox). Once I read the
mb2md source code and understood this, I used absolute paths and all was well.

* Rich Pieri also suggested remailing the messages to the IMAP server
using formail. I have not tried that, hoping that offlineimap or mbsync
will do the trick.

Thank you again!!!

--
Dan Barrett
dbarrett at blazemonger.com