[Discuss] No-SQL Database Recommendation?

[kentborg at borg.org (Kent Borg)]

On 01/11/2015 09:12 PM, John Hall wrote:
> I'm not convinced you need a database. Writing your own simple 
> persistence layer can be really easy and reliable with Python.

Interesting suggestion, rolling my own is tempting, and when someone 
else suggests it...the temptation rises. But the multiuser part is a 
pain, I am happy to let someone else invent and implement that for lazy 
old me. Assuming that really is the lazy way out...

> I'm curious about what the intended application is,

Until my project is closer to reality, I would like to avoid specifics. 
Certainly I am looking forward to getting things working well enough to 
demo, believe me.

> the OS distribution you are using, hardware and other loads on that 
> hardware.

So far I have been playing on Ubuntu because it is handy. (Though lately 
I have been losing patience with Ubuntu and preferring Debian. You know, 
prejudices and aesthetics again.)

> I'm concerned that your Mongo issues might be due to some system 
> bottleneck that you might run into with another solution as well, 
> anyway, try using "nice" to prioritize your processes and "ionice" to 
> make sure background processes and cron jobs don't dominate your disk io.

The fact mongodb was proudly telling me it found a half million records 
made me think it was spending at least some time to even know it found a 
half million records--where I only want a few records, and want to get 
those few records appropriately quickly. I looked for some tuning option 
to not do that work, but didn't find one.

> I assume you've heard about SQLAlchemy

Yes, I think I have heard the name before, but that's all. On your 
recommendation I have looked it now: It looks nice! Quite Pythonic, I 
think it will let me aim my brain at other stuff again.

I think my immediate todo turns into something close to:

   - install postgresql and sqlalchemy

   - create my user

   - fire up python to create my tables, index my time field, and a 
stuff in bunch of fake data (looks like the wikipedia article is a nice 
enough cheat sheet to make me dangerous)

   - see if I can figure out how to do my "gimme just a few of the many 
hits" limited query, and see how fast it is

I suspect Postgresql will handle it well, quick Googling suggests the 
minimal RAM foot print is plenty small for me, and if it turns out I 
later *do* want any replication or relational stuff, Postgresql has 
those features.

> I agree with others that if you need a database PostgreSQL would be a 
> fine solution and with Richard Pieri's original litmus tests on if you 
> should consider a nosql database.

SQLAlchemy looks like it will shield me from the SQL stuff I dislike. 
And if I must craft a few SQL statements for debugging purposes, I can 
do that, I can be tough and face the music, if I have to. I'm just a 
whiny brat who is spoiled by how nice Python is.

(Python really is wonderful. Trying out some new library is easy and I 
can start writing code quickly. By the time I have really understood 
what I precisely want to do...I mostly have a working program. Maybe not 
final, but working. Certainly Python has flaws, such as having an adhoc 
language definition and being a big, monolithic, single-threaded 
interpreter, but it is nice anyway.)

Thank you for your thorough suggestions,

-kb