[Discuss] time-warp

[worley at alum.mit.edu (Dale R. Worley)]

Bill Bogstad <bogstad at pobox.com> writes:
> A few questions though:
>
> 1. How does the archive size compare to the 'source' directory?

Let's see, I've never really tallied things up.  The source breaks down:
    on the main partition: 11G
        of which a handful of large files/directories are excluded:  7G
        the rest are generally subject to backup:  4G
    on another partition, the "not replicated" stuff:  49G

I do keep clones of the IETF's RFC directory and all of the
internet-drafts going back years, but they're on the not-replicated
partition.  I have other saved documents, but they're 300M.  Similarly
for media, I've got a lot of images but few audio and video files, so it
all comes to 1.2G.

> 2. Do you ever run 'git gc'?  (I would expect that to possibly reduce the size
> of the archive.)

Weekly, I run the script that clips out commits that are now "too
closely spaced" for their age.  That recreates the chain of "commit"
objects in the Git repository, though of course the commit objects in
the new chain reuse the "file tree" objects of the commit objects in the
old chain that they were copied from.  So the entire old chain of
commits, the file tree objects of the excised commits, and the "file"
objects for the versions of files which are no longer recorded become
garbage.  You've got to purge the garbage regularly, of course, or the
archive grows linearly with time.

So immediately after that, I run "git gc", then copy the Git archive and
make an ISO image of the copy.

> 3. I keep things like computer standards documents (only 4.5 Gig :-),
> photos/videos of friends and family, ebooks, as well as git trees of
> software that interest me underneath my home directory.  If you have
> anything like that, you clearly aren't fitting that into 3.4Gig.  Any
> thoughts on how you would handle stuff like that?

One oddity is that Git will detect a Git repository that is within the
working tree of the Git repository you're operating on and refuse to
consider the sub-repository as contents of your repository.  But in
practice, I don't have a lot of media or documents (the big things are
clones of IETF things), I don't do a lot of software development (so I
don't care much about Git trees).  The big things are excluded with
.gitignore or put into the second partition (which isn't a subtree of
$HOME).  The one exception is archival enough that I've burned a
separate backup DVD of it and that's sufficient.

>> I can send you a copy of the scripts if you want to use time-warp.
>
> In any case, I would still be interested in seeing a copy of the scripts.

OK, I'll send you a copy.

Dale