LVM + RAID follow up

[warlord at MIT.EDU (Derek Atkins)]

Quoting John Abreau <john.abreau at zuken.com>:

>> Now, I want to swap out these 80G drives with 200G drives...  How
>> would you do it?  I dont understand what you mean by "swap two drives
>> at a time." -- in all cases I don't see how you can swap more than
>> a single drive at a time, let the raid rebuild, and then move on to
>> the next one..  Could you walk me through your procedure?
>
> Maybe I shouldn't have used the word "swap" there. What I meant by
> "swap two drives at a time" is to add the two 200G drives, then migrate
> the data off the two 80G drives and onto the two 200G drives, then
> remove the two 80G drives.

So what you are saying is that you now have four drives in your system
during this upgrade?  I'm planning to max out my system so I can't do that.
That could be part of the disconnect here -- I was trying to update the
disk array in-place, not have two arrays and migrate from one to the
other.  I dont have the physical space to double the number of disks
attached to the system.

> Two other things: I like to include swap within LVM, rather than as
> a separate partition. Also, since md0 is small, I assume that's
> meant for /boot, and /boot cannot be within LVM.

Yeah, I know, no /boot in LVM.  /boot can't be RAID-5, either, only
RAID-1.  As for SWAP in LVM...  *shrugs*  I think it's higher performance
not to have swap in RAID or LVM.  You can always add multiple swap
partitions, and if you have a disk failure it's easy to just 'swapoff'
your bad disk.

>> I guess that now that pvresize is implemented it might be easier,
>> but that's only been implemented since FC5.
>
> The HOWTO did mention that there are two versions of LVM, and that
> volumes created with lvm1 can be incompatible with lvm2.  I only
> saw it mentioned in passing when reading the section on LVM snapshots,
> so I'm not sure what all the differences are between the two versions.

I'm only talking about LVM2.  I wasn't even trying to think about
LVM1.  But the pvresize function hasn't been implemented (even in
LVM2) until recently!  On FC4:

rpm -q lvm2
lvm2-2.01.08-2.1
/usr/sbin/pvresize --help
  pvresize: Resize a physical volume in use by a volume group

Not implemented.  Use pvcreate options.

But on FC5:

rpm -q lvm2
lvm2-2.02.01-1.2.1
/usr/sbin/pvresize --help
  /etc/lvm/.cache: open failed: Permission denied
  pvresize: Resize physical volume(s)

pvresize
        [-d|--debug]
        [-h|-?|--help]
        [--setphysicalvolumesize PhysicalVolumeSize[kKmMgGtT]
        [-t|--test]
        [-v|--verbose]
        [--version]
        PhysicalVolume [PhysicalVolume...]

> If you choose a RAID component size of, let's say, 40G, then your 80G
> drives would each have room for 2 RAID components, and your 200G drives
> would each have room for 5 RAID components. We'll forget about /boot
> for now, to simplify the picture.

Yeah, but then a few years from now if I get 1.2TB drives that would
require 30 partitions.  Can you even HAVE that many partitions on a
drive?

> You could add in the 200G drives one at a time, which gives you 5 of
> your 40G partitions. Then raid-fail an 80G drive, and replace its two
> partitions with two of the five partitions on the 200G drive.
>
> So what we have in this scenario is two existing RAID-5 metadevices,
> and with the additional three 40G partitions on the new 200G drives,
> we create three more RAID-5 metadevices. Then we add the new
> metadevices to the LVM VG.
>
> Alternately, you could just stick with an 80G partition and a 120G
> partition, but using partitions all of the same size will make
> the upgrade procedure less complicated, particularly the next time
> you want to upgrade the disk sizes.

Well, if I'm going to swap out all the drives at the same time it's
easy:  I just make the raid-5 base partition the full size of the
drive; swap out the drives one at a time (and let the raid rebuild
between each swap), then once all the drives are swapped out I can

  mdadm --grow ...
  pvresize ...
  lvextend ...
  ext2online ...

The only time this process doesn't work is if I want to swap out only
a subset of my raid5 disks...  Or to use the extra space taken by the
/boot RAID-1 partitions on the drives that aren't on the hda/hdc drives.

Anyways, I think I've got my answer now -- you're talking about doing
something that I can't do -- so I just need to think about replacing
one disk at a time and them upgrading the raid configs once all the drives
have been replaced.

Rich Braun wrote:

> Derek is that how you're actually setting it up, your swap is *not* 
> configured
> as RAID?  I've always assumed that if you don't want to have your system go
> down upon an hdd failure, you really want swap to be sitting on top of RAID.
> (I had it as its own /dev/md1 partition until I started using LVM to assign a
> volume as swap along with the various filesystems.)

Yeah, that's how I have it configured.  It's much better performance
to do it this way (why double the number of writes you have to make when
you swap out a page?)..  And if a drive fails my mdadm monitor and logwatch
will tell me long before the system comes to a halt and I can just swapoff
that partition.

> If you do this, of course, at init you need to run
>
> # /sbin/mdadm --monitor --delay=300 --scan --daemonize
>
> so you'll get an email whenever a RAID array degrades.  Otherwise you'll go
> months before realizing a drive's bad.

Yep!  I do that.  :)

Thanks,

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.