[Discuss] Stalwart email & DB vs file system for message storage

[ron at bclug.ca (Ron)]

markw at mohawksoft.com wrote on 2026-01-21 05:52:

> A database makes sense for the indexing and organization of email,
> but storage for the files itself makes no sense.

For what it's worth, with Stalwart there are several different stores:


Internal


Data (metadata, folders, various settings - essentially all except blobs)


Blobs (emails, sieve scripts, other files)


Search (full-text search)


In-Memory (key-value storage for SMTP server, anti-spam, etc.)



The default storage for everything is RocksDB, a file-based system
that's not one single file like SQLite but a few files like so:

-rw-r--r-- 1 root root 179768 2026-01-22 01:03 000004.log
-rw-r--r-- 1 root root     16 2026-01-22 00:27 CURRENT
-rw-r--r-- 1 root root     36 2026-01-22 00:27 IDENTITY
-rw-r--r-- 1 root root      0 2026-01-22 00:27 LOCK
-rw-r--r-- 1 root root 333787 2026-01-22 01:07 LOG
-rw-r--r-- 1 root root   1468 2026-01-22 00:27 MANIFEST-000005
-rw-r--r-- 1 root root 132563 2026-01-22 00:27 OPTIONS-000007


I suspect its performance is better than SQLite, else Google (LevelDB) 
then Facebook (RocksDB) would've just used SQLite.


Sometimes it's easier to fork something than to try to convince someone
else to merge your changes if they don't value the features.


Cheers




And I remain unconvinced that a DB isn't well suited to email messages
since it works for Google's Gmail.

I'd need to see some *evidence* that it isn't a good idea.



Stalwart won the Github funding prize so they'll be presenting at
FOSSDEM this year, hopefully with results from their attempts to scale
up to "Gmail scale" with prize money spent on cloud infrastructure to
test with millions(?) of users connecting up.

> Stalwart: Can Open Source do Gmail-scale Email?
> 
> Ever wondered how Gmail, Yahoo, and Apple iCloud manage to host
> hundreds of millions of email accounts reliably? How do they store
> petabytes of messages, survive hardware failures without losing
> data, and keep spam at bay across billions of daily deliveries?

https://fosdem.org/2026/schedule/event/FP8EFX-scaling-email/


Could be interesting.