[Discuss] easy clustering of applications

[bogstad at pobox.com (Bill Bogstad)]

On Wed, Apr 2, 2014 at 7:02 AM, Edward Ned Harvey (blu)
<blu at nedharvey.com> wrote:
>> From: discuss-bounces+blu=nedharvey.com at blu.org [mailto:discuss-
>> bounces+blu=nedharvey.com at blu.org] On Behalf Of Tom Metro
>>
>> Edward Ned Harvey (blu) wrote:
>> >>Tom Metro wrote:
>> >> It does seem like every application has its own unique approach to
>> >> clustering. There is no generalized solution.
>> >
>> > This is one of the reasons why I'm much more strongly inclined toward
>> > HA at the hypervisor. ...if you do HA at
>> > the hypervisor, then it doesn't matter what your application is; you
>> > have HA of your application, no matter what it is, and the setup is
>> > identical for any and all types of application.
>>
>> I don't think what you are describing is possible, except for a certain
>> class of applications, unless you are glossing over some of the
>> application-specific glue that gets wrapped around your VM.
>
> I assume other hypervisors can do this too, but vmware is the one I know.  Vmware High Availability Clustering is able to maintain the guest machine internal running state across multiple heads.  They do it by establishing a few control points around IO.  They snapshot the CPU and RAM state at a select moment and synchronize across both heads, and when they reach another checkpoint (such as sending network packet, or disk IO packet) then they send the internal CPU/RAM state snapshot differential to the other head.  Since they're aggregating lots of CPU cycles into a single checkpoint, it doesn't hurt performance too much, and since they're controlling IO chokepoints, if you need to failover there is no state inconsistency or miscommunication with clients.  So for all intents and purposes, both machines are always in lock-step internal state with respect to CPU and RAM.  If one head fails, the other head simply continues.

I can see how that would work, but it seems to me that the cost of
having the VM do this could be very high depending on the application.
  An application that does little IO, has a high memory footprint,
and modifies all of it between IO requests would make for very
expensive checkpointing.   Every checkpoint could require transferring
multiple gigabytes of modified RAM.   A CPU can dirty RAM way faster
then all but the fastest network connections can transfer it.

Bill Bogstad