[Discuss] Redundant array of inexpensive servers: clustering?

[richb at pioneer.ci.net (Rich Braun)]

It's 2014 and I figured that maybe the state of the art in RAIS (true
clustering of servers vs. disks) might have gotten somewhere since the last
time I looked at the idea in about 2011.

I have two home servers (down from 3, the electric bills are punitive where I
now live) with a dozen services running on them. The only true clustering
technology I've gotten working so far is Galera+MySQL (and it actually
requires a third "server", which I put on an Acer Revo sitting in a corner to
protect against split-brain).

The problem with other services is they either don't have clustering
capability or are a true pain to set up for clustering.  (Think postfix local
delivery, think Jira, think MythTV's backend.)

One of my coworkers suggested GlusterFS but that's a piecemeal approach which
might cluster one or two of the apps, and he warns me that it'll fall apart in
some scenarios.

My goal is simple:  mash the power button or yank the network cable from
either of these machines, and have all the apps still running.  Then plug the
machine back in and have all state restored to full redundancy without having
to type any commands.  (Galera actually accomplishes this, it's a dream come
true!)

I'm posting this after a frustrating 24 hours with old technologies AoE (ATA
over Ethernet) and OCFS2 (Oracle's clustered filesystem).  They have the feel
of  something you'd download off ftp.uu.net sometime around 1997 and spend a
week or three getting to work.  OCFS2 is horrible; AoE comes up OK but by
itself accomplishes nothing beyond a collection of universally accessible raw
disk blocks.  Turning those blocks into HA email on my screen or a monitoring
system that doesn't go down is an exercise for the reader.

Is HA a dead technology?  I'm flummoxed that the state of the art still feels
so 1990s.  What am I missing?  Did the commercial cloud services co-opt all
development and horde this technology for their own overpriced business
models?

-rich