[Discuss] SSH options (was Future of X11 (was Trying to connect to internet in Debian))

[markw at mohawksoft.com]

> Rich Pieri wrote on 2026-01-20 18:57:
>
>> On Tue, 20 Jan 2026 14:05:10 -0800 Ron<ron at bclug.ca> wrote:
>>
>>> A DB makes most sense for email messages - index-able, ACID
>>> compliance (depending on DB), fast, etc.
>>
>> Databases are the WORST possible way to store arbitrary files.
>
> We're talking email *messages*, not files - there's no requirement that
> messages are files.

What is a "file?"

Generally speaking, a database isn't a good place to store arbitrary data.
Most database designs are built on a notion of "blocks" and the cache is
based on accessing blocks. Databases like PostgreSQL have facilities to
expand data storage beyond the block sizes, in pgsql, this is TOAST.

"Databases" (This is a discussion in and of itself, but I won't get into
it right now) like "records" which are finite pieces of data that can be
indexed or correlated with other pieces of data, i.e. relations.

>
>
> And when it comes to storing data, databases are kinda designed for it.

Not really true. Databases are designed around "accessing data" not so
much storing it. Yes, "storing data" is important, but arbitrary and
arbitrarily sized data isn't what a "database" is good for. That's the job
of file systems.

>
>
> Email is data: each message a record; maybe a column for headers,
> subject, to, from, and body, folder, date,...
>
> Index several of those for fast access. Perfect.
>
>
> Postfix and Dovecot both already support storing & retrieving info with
> SQL.

There is no problem what so ever with using a database for these things,
but the message body, all those embedded jpegs, videos, viruses, and so on
should remain in the file.
>
>
>
>>> when a message is marked "Read", the *file name* is changed to
>>> reflect the change in status!
>>>
>> It's efficient
>
> Maybe the rename is efficient but...

but?
>
>> It's fast, faster than any database
>> could ever hope to be on the same filesystem.
>
> Not a chance, else PostgreSQL et al would store each record as a file.

As a former PostgreSQL committer and former database architect, I will
tell you that you are wrong. PostgreSQL is one of the best databases there
is, but it probably can't beat renaming a file because renaming a file is
a simple operation. In PostgreSQL, if you have a field as "unread" and
then update the field as "read," the data in the block cache is updated
and written to the database file as a new block/row. I won't get in to how
transaction IDs relate to MVCC visibility. There are multiple writes
involves with an update that isn't needed for renaming a file.

PostgreSQL like any other database manages data differently than file
systems and sacrifices performance for durability.

>
> "Find the one message out of 50,000 where the subject is $x" is going to
> be faster in a DB. Every. Single. Time. (Searching on the server is not
> uncommon.)

This is basically not true either. If you have an index on a field, you
are kind of limited in matching ability. Unless you have some nifty tricks
(Which I have a whole library of) you will either be using an index where
you match the string of the field or do a full table scan. An arbitrary
text search in a SQL database results in a full table scan unless you have
some specific text search software.
>
>
> Also, it depends on which file system it's installed on. Some are tuned
> for many small files, some less so.

This is true. ReiserFS was really quite good for mail servers, it just
killed it. :-)

>
> And running out of inodes can be rather disruptive.

Use ZFS and its not an issue.

>
>
>
> Per the Stalwart dev(s):
>
>  > Maildir is an ancient format that does not scale well. It is not going
>  > be supported [as a message store] by Stalwart.
>
>  > anything beyond exporting/importing Maildir is out of scope for
>  > Stalwart. Making Stalwart write Maildirs to the filesystem would feel
>  > like adding an exhaust to an electric car
>
>
>
>
>> It's brilliant in its simplicity.
> But it's outdated in its implementation. This isn't the 90s anymore
> (MailDir designed in 1995).

I'm not a fan of "it's old" are an argument. What is bad about it? Also,
systemd uses file system stuff and you don't think that's bad. Why the
disconnect?

>
>
> And, what other software changes filenames to indicate something has
> happened with the file?
Why does this matter?

>
> myfile.txt becomes myfile.txt.read once opened? Nonsense.

Why?
>
>
> I say this as someone who has a script that relies on Maildir files to
> rewrite the subject of my Chinese Word Of The Day lessons to include the
> kanji & jyutping. Not sure how I'd do that in a DB.

Easily?
>
>
> Oh, and backups - rsync sees all these changes in file names and wants
> to back them all up again despite the contents unchanged.

It would backup the WHOLE database table file and index on every change
anyway. So what?

>
> PostgreSQL will happily replicate only changed records offsite for
> backup, maybe a bit more flexible.

Only if you set it up correctly. It won't do it out of the box. You would
probably use WAL replication, but yawn. What you REALLY want is a ZFS file
system which will only send changed blocks with "zfs send."
>
>
>>> Designed to be *fast* at up-to a couple hundred terabytes.
>>>
>> Designed to compete with Microsoft Exchange on Windows, more like.
>
> So?
>
> I'm unclear on what that's supposed to mean, but since it's *not* MS
> Exchange, it *does* support open standards, and it *is* open source
> software, what's the problem?
>
>
> Also, when scaling beyond "my own personal email server, plus maybe
> family" to "people I do not know pay money for email accounts", Stalwart
> allows users to do their own password management.
>
> Stalwart also allows user-controlled at-rest encryption of email.

Why not use LUKS?
>
> Those are quite important these days.
Personally I would not trust application level encryption without a
careful audit. People generally do not understand encryption and almost
always do it wrong.

>
>
> One can run RoundCube webmail client with necessary plug-in to interact
> with Dovecot's password management to allow users to change their own
> passwords, but that requires PHP, RoundCube, httpd, etc.

Passwords? PHP? Yikes.
>
>
> And still no encryption at-rest (with per-user encryption keys).

LUKS or better yet, SEDs (selfencrypting drives). No one in the industry
trusts application level encryption because it is generally bad.

>
>
> Finally, it's nice to have quotas for mailboxes, Cal/Card/WebDAV managed
> in one place, i.e. one quota applied to everything.

Linux has quotas built in.

>
>
>
>> Stalwart runs on Windows which means it needs to perform acceptably
>> on NTFS, and the "easy" way to do that is throw it all in a big ol'
>> database.
>
> Got anything to back up that assertion?  Because "Stalwart devs chose a
> database because Windows" sounds like some wild fever-dream conspiracy
> theory.
>
>
>
> Both mbox and maildir suffer from inefficiencies of either large files
> (mbox) or many small files (maildir).  Both suck for efficiency when
> dozens / hundreds / thousands / more users are accessing storage of
> possibly millions of messages.

Large files are not bad and many small files are not bad if hierarchically
stored.
>
>
> Databases are a perfect tool for fast, concurrent access.

Not really for reasons stated.

>
>
>
>
> Here's an example MailDir file name:
>
> 1768962755.V2c00b601I89a3bM177540.mail.bclug.ca:2,RSb
>
> Probably "Read", "Seen", plus some other flags set - mark as unread or
> reply and whole file gets backed up again.

OK? So?
>
> And, that's not even touching on how folders are represented:
>
> .Archives.2026.01_January
>
> is roughly the folder path for /Archives/2026/01_January (I've forgotten
> that particular piece of brain damage).

Again, I don't see the issue. Human readability isn't important.
>
>
>
> I'm fond of MailDir because I have a Python script on the server that
> detects new messages in a specific folder, it parses the HTML, rewrites
> the Subject, recomposes the message so Subject lines are meaningful,
> then writes the new file to ./INBOX/Cantonese/new and voila, I can
> refresh my Chinese learning from scanning subjects.
>
> That'll be more work to do if I decide to use Stalwart for email.
>
>
> Almost certainly will use it for Cal/Card/WebDAV though - that's another
> critical feature that Postfix / Dovecot don't support, requiring
> something else (NextCloud or Stalwart seem the best options?).
>
> _______________________________________________
> Discuss mailing list
> Discuss at lists.blu.org
> https://lists.blu.org/mailman/listinfo/discuss
>