Re: ATA over Ethernet (aoe) Experience?

["peter.petrakis (Bugzilla)" <user=1616@nabble.blu.org>]

 The storage failover scenario you've describe is what DRBD excels at, 
http://www.drbd.org; think of it as raid 1 over the network. Now when 
it comes to VM migration, that call is really up to you and how much 
work you want to put into the scripting. With DRBD running, the I/O 
will be re-routed to the secondary node automatically which should 
give you plenty of breathing room to replace the failed disk. If you 
want a product that does all this for you, I would recommend this: 
http://www.stratus.com/products/avance/overview.htm. I'm a little 
biased here since I work on both projects :) 
Now when it comes to failing disks in general, just try to avoid SATA 
and stick with SAS. Yes it costs more but in my opinion the uptime you 
get in return (and peace of mind) is worth it. I Hope this helps. 

Peter 

On Sat, Jul 12, 2008 at 7:07 PM, Kent Borg <[hidden email]> wrote: 
> Anyone here have experience with ATA over Ethernet ("aoe")? 
> 
> To be specific, here is my idea: I want to do more of that trendy 
> "server consolidation" but not have all my eggs in one basket.  So I 
> imagine having two identical physical machines, each set up to match the 
> other.  Normally I run all the VMs on one machine.  Worrying about disks 
> dying I put each VM on a pair of software raid 1 disks in that box.  But 
> how do I quickly switch to the other physical machine? 
> 
> Here is where I get clever (and ask for advice):  On the other machine I 
> do the exact same thing.  VM on a pair of software raid 1 partitions. 
> But how to sync them up? 
> 
> First, shut down the spare copy.  Only run one copy of the VM at a time. 
> 
> Second, shutdown the md device for that spare copy. 
> 
> Third, export the two physical partions via ATA over ethernet.  (Be an 
> aoe target.) 
> 
> Fourth, on the primary machine, import the two partitions via ATA over 
> ethernet.  (Be an aoe initiator.) 
> 
> Fifth, on the primary machine, add the two partitions to the raid 1 md 
> device.  Wait for the data to copy, but still keep the VM running and 
> serving whatever good stuff it does. 
> 
> 
> I haven't researched it much, but if I have dedicated gigabit ethernet 
> )or faster) and a crossover cable, and do a large MTU hack on those two 
> NICs, maybe the ATA over ethernet could mostly keep up with the local 
> disks?  Or maybe not... 
> 
> Next obvious idea would be to not ship both partitions over the ethernet 
> cable separately, but to put them together into a raid 1 pair first and 
> instead ship the md device over the cable. 
> 
> But there is something else I would like to do: run the VM on one 
> physical machine, shut it down, reverse the ATA over ethernet setup, 
> boot the VM on the other physical machine, and have the various md 
> components all happy and in synch with no replication necessary.  Doing 
> a flat collection of 4 distinct physical partitions would probably 
> work.  But if I did nested and rehanging the tree, a complete 
> replication would have to happen unless I did some hacking. 
> 
> Example. 
> 
> case 1: 
>  - yin's sda1 
>  - yin's sdb1 
>  - yang's aoe, which is 
>    - yang's sda1 
>    - yang's sdb1 
> 
> case 2: 
>  - yang's sda1 
>  - yang's sdb1 
>  - yin's aoe, which is 
>    - yin's sda1 
>    - yin's sdb2 
> 
> Is there a way to hack md internals to make the two cases boot back and 
> forth and keep the md in synch both ways? 
> 
> 
> Practical consession: maybe just running in this aoe mode while messing 
> with configurations and migrating VMs, and just run local raid 1 when 
> things are stable. 
> 
> 
> Comments? 
> 
> 
> Thanks, 
> 
> -kb 
> 
> -- 
> This message has been scanned for viruses and 
> dangerous content by MailScanner, and is 
> believed to be clean. 
> 
> _______________________________________________ 
> Discuss mailing list 
> [hidden email] 
> http://lists.blu.org/mailman/listinfo/discuss
>