[Discuss] lvm snapshot cloning

[Edward Ned Harvey]

> From: discuss-bounces+blu=nedharvey.com at blu.org [mailto:discuss-
> bounces+blu=nedharvey.com at blu.org] On Behalf Of
> 
> Suppose you have a 1TB hard disk. How on earth do you back that up? If you
> use a 1TB USB hard disk (USB 2.0) At best case, you'll get about 30MB/s.
> (You won't get that fast, but it is a good round number for discussion)

Yes, what you need is something that does incremental snapshots.  Presently
zfs, soon btrfs, and I heard you say it can be done with LVM although I've
never seen that done.  Also, I think there's probably a solution based on
VSS.  And of course WAFL and other enterprise products.

Additionally, there are lots of "potential" solutions, for example, use FUSE
to mount a filesystem, and because it's running through fuse, you'll be able
to keep track of which blocks change and then you can perform optimal
incrementals etc.  But I'm not aware of anything that is actually
implemented this way.


> 1TB is 1,099,511,627,776 bytes. At 30MB/s that's a little over 9 1/2 hours
> to backup.
> 
> If you use a dedicated high speed SATA or SAS drive, you may get a
> whopping 160MB/s. That's 1 hour 49 minutes.

Take it for granted, you won't be stuck at USB2 speeds.  USB has universally
(to the extent I care about) been replaced by USB3, since about 2-3 years
ago.  The bottleneck is the speed of the external disk, which I typically
measure at 1Gbit/sec per disk.  (You can sometimes parallelize using
multiple disks depending on your circumstances.)  It's random IO that really
hurts you.


> With snapshots, you can back up a consistent state, as if your 2 hour
> backup happened instantaneously, but you are still writing a lot of data.

Right.  Presently at work we have a bunch of ZFS and Netapp systems, that
replicate globally every hour, and use various forms of WAN acceleration,
compression and dedup.  Although it is optimal, it is very significant,
noticeable strain on the WAN.