[Discuss] Stalwart E-mail; Was: Re: SSH options

[slitt at troubleshooters.com (Steve Litt)]

Ron said on Fri, 23 Jan 2026 18:57:21 -0800

>Rich Pieri wrote on 2026-01-22 08:02:
>
>> You dropped the "arbitrary". When everything fits neatly into
>> tables or key/value stores then sure, a database might work. Email
>> messages are not neat. They are very much like medical records:
>> arbitrary in size and structure. Few databases can deal well with
>> this kind of data.  
>
>Quite a few actually *can* deal with this kind of data; it's been a
>fairly mature field for a decade or more already.
>
>
>You can read about databases designed to work with unstructured data at
>your leisure:
>
>https://en.wikipedia.org/wiki/NoSQL
>
>
>> You can read up on all of Oracle's failures in the EMR space, and 
>> the history of WinFS, at your own pace, as examples of this.  
>
>So, two companies that are bad at software had failures, so in this
>field a database is doomed to failure?  Disagree.

You're right: Two companies' failure doesn't prove it's bad. But
failure of two such well financed companies is evidence that it's
difficult to get right, leading a reasonable person to suspect (not
know, suspect) that even if successful, it just might have problems
that pop up in edge cases.

[snip]

>With the goal of competing on a scale of
>
>> to host hundreds of millions of email accounts reliably? How do they
>> store petabytes of messages, survive hardware failures without 
>> losing data, and keep spam at bay across billions of daily 
>> deliveries?  
>
>the thought of billions (or millions) of Maildir files per day is
>laughable.

The preceding sentence is an ad homonym logical fallacy. It's your
opinion, which might be right, might be wrong, or might be "it depends".

To me the dead bang obvious course of action is to keep emails in
individual files, in a directory tree so that no single subdirectory
contains more than 1000 files, and then keep a copy of the email's
subject, date, to, from, cc, bcc, List-ID and full pathname of the email
in a relational database like Postres or MariaDB. Now you can lightning
quick look up by all major headers, but still have the email message
stored as an email file readable from any email client. If you simply
*must* be able to search bodies, you can have a words table that can be
joined to the emails table. J Timberlake how heavy is that?

SteveT

Steve Litt 

http://444domains.com