Backup Question, Preserving Hard Links

[kentborg at borg.org (Kent Borg)]

I am looking for a way to:

 1. Copy all the files from one disk to another, preserving all
    permissions, ownership, etc.;

 2. don't quite copy ~all~ the files, exclude some files/directories
    by pattern;

 3. preserve hard link relationships (i.e., if both ~/a and ~/b are
    hard links to the same data, copy only the data only once, with
    two links pointing to it);

 4. be incremental (if the source and destination files have the same
    name and date and size and generally look the same, assume they
    are the same and don't copy them again);

 5. be incremental (if an old file on the source disk goes away,
    delete it from the destination disk);

 6. be incremental (if a new hard link appears, say ~/c, pointing to
    the same file as last week's ~/a and ~/b, preserve the new
    three-way relationship in the destination);

 7. handle lots of data and lots of hard links; and

 8. be fast.


The obvious solution is to use rsync, but I have too many files for
that.  Rsync takes forever and far too much memory just to figure out
what is it going to do.  I maxed out my motherboard at 3 GB and rsync
still takes all my excessive amount of swap (yes, this is a known
bug/design festure of rsync).  My current approach is to delete the
destination completely, and then pipe two tars together, a -C on the
first to set the source, and a -C on the other to set the destination.


There has to be a better way.  What is it?


Thanks,

-kb, the Kent who is copying 300 GB into a USB 2.0 encrypted loopback,
and trying not to move it all every backup, because it takes all day
to do so.