LVM/RAID recovery question

[Matthew Gillen]

Rich Braun wrote:
> Issue:  the RAID1 superblock for /dev/md1 is blown away on /dev/sda2
> 
> Observation:  the /dev/md1 device was coming up on /dev/sdg2 but the LVM was
> (silently until I started scrutinizing this bizarreness more closely) running
> the live system on /dev/sda2 instead of /dev/md1.
> ...
> It surprised me that the LVM would come up at all on a partition that had been
> zapped this way,

The nice thing about sw RAID in linux is that it writes the superblock 
at the /end/ of the device.  So if something is just looking at the 
first few blocks of the device, they wouldn't know that it's a RAID 
device.  So I'm not surprised that you were able to mount and read data 
off the device as if nothing happened, but I am intrigued about how it 
switched itself over from looking at /dev/md1 to /dev/sdg2...

In fact, in a story that's sure to horrify some of the sys-admins on 
this list, once I even took an existing ext3 filesystem and made it a 
RAID device (without losing the data).  This was before support for 
shrinking ext3 filesystems, so I just made sure not to fill up that disk 
(since technically the filesystem thought it still owned the last few 
blocks on the device, which is where the RAID superblock was...I did 
eventually migrate off that RAID device onto a properly constructed one 
without getting shot in the foot).

It looks like you might be in better shape though, since you already 
have the space for the superblock properly set aside.  So the good news 
is that it's probably possible.  The bad news is that RAID isn't enough 
sometimes: it seems that lack of organization and life priorities have 
led me to purge some old email where I had detailed exactly what I did. 
  It's also been a few years, so the details might not have been useful 
anyway.

Back to actually helping you though, check questions #11 and 12 of this 
FAQ for something that might help you (re-creating superblock):
http://svn.debian.org/wsvn/pkg-mdadm/mdadm/trunk/debian/FAQ?op=file&rev=0&sc=0

> Or perhaps some alternative method of telling the
> LVM to shuffle disk blocks around to a spare drive without having to resort to
> backup/restore?

This might be easier / less prone to lose data.  Add a device to your 
LVM, then do a 'pvmove' (the man page has a good example).  Then remove 
your broken "RAID" device from the LVM and re-create it properly, add it 
back to LVM and 'pvmove' back.  I think you can even do all that without 
taking the system offline.

Good luck.
Matt