Boston Linux & UNIX was originally founded in 1994 as part of The Boston Computer Society. We meet on the third Wednesday of each month at the Massachusetts Institute of Technology, in Building E51.

BLU Discuss list archive


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Discuss] No-SQL Database Recommendation?



On 01/11/2015 09:12 PM, John Hall wrote:
> I'm not convinced you need a database. Writing your own simple 
> persistence layer can be really easy and reliable with Python.

Interesting suggestion, rolling my own is tempting, and when someone 
else suggests it...the temptation rises. But the multiuser part is a 
pain, I am happy to let someone else invent and implement that for lazy 
old me. Assuming that really is the lazy way out...

> I'm curious about what the intended application is,

Until my project is closer to reality, I would like to avoid specifics. 
Certainly I am looking forward to getting things working well enough to 
demo, believe me.

> the OS distribution you are using, hardware and other loads on that 
> hardware.

So far I have been playing on Ubuntu because it is handy. (Though lately 
I have been losing patience with Ubuntu and preferring Debian. You know, 
prejudices and aesthetics again.)

> I'm concerned that your Mongo issues might be due to some system 
> bottleneck that you might run into with another solution as well, 
> anyway, try using "nice" to prioritize your processes and "ionice" to 
> make sure background processes and cron jobs don't dominate your disk io.

The fact mongodb was proudly telling me it found a half million records 
made me think it was spending at least some time to even know it found a 
half million records--where I only want a few records, and want to get 
those few records appropriately quickly. I looked for some tuning option 
to not do that work, but didn't find one.

> I assume you've heard about SQLAlchemy

Yes, I think I have heard the name before, but that's all. On your 
recommendation I have looked it now: It looks nice! Quite Pythonic, I 
think it will let me aim my brain at other stuff again.

I think my immediate todo turns into something close to:

   - install postgresql and sqlalchemy

   - create my user

   - fire up python to create my tables, index my time field, and a 
stuff in bunch of fake data (looks like the wikipedia article is a nice 
enough cheat sheet to make me dangerous)

   - see if I can figure out how to do my "gimme just a few of the many 
hits" limited query, and see how fast it is

I suspect Postgresql will handle it well, quick Googling suggests the 
minimal RAM foot print is plenty small for me, and if it turns out I 
later *do* want any replication or relational stuff, Postgresql has 
those features.

> I agree with others that if you need a database PostgreSQL would be a 
> fine solution and with Richard Pieri's original litmus tests on if you 
> should consider a nosql database.

SQLAlchemy looks like it will shield me from the SQL stuff I dislike. 
And if I must craft a few SQL statements for debugging purposes, I can 
do that, I can be tough and face the music, if I have to. I'm just a 
whiny brat who is spoiled by how nice Python is.

(Python really is wonderful. Trying out some new library is easy and I 
can start writing code quickly. By the time I have really understood 
what I precisely want to do...I mostly have a working program. Maybe not 
final, but working. Certainly Python has flaws, such as having an adhoc 
language definition and being a big, monolithic, single-threaded 
interpreter, but it is nice anyway.)

Thank you for your thorough suggestions,

-kb



BLU is a member of BostonUserGroups
BLU is a member of BostonUserGroups
We also thank MIT for the use of their facilities.

Valid HTML 4.01! Valid CSS!



Boston Linux & Unix / webmaster@blu.org