RAID5 Online Resizing with Linux
Today I wanted to test the raid5 reshape feature of Linux kernel >=2.6.17. My testing environment was VMware with 3 virtual disks each containing a partition of 200MB which should be resized to 500MB, using grml (featuring kernel 2.6.17-grml) of course. For real-live setup adjust the steps for stuff like switching harddisks of course. ;-)
# Initiate a RAID5 setup for testing purposes: mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/hda1 /dev/hdb1 /dev/hdd1 # Create filesystem, mount md0, create a testfile and save md5sum for later check: mkfs.ext3 /dev/md0 mount /dev/md0 /mnt/test dd if=/dev/urandom of=/mnt/test/dd bs=512 count=10000 md5sum /mnt/test/dd > md5sum # Make sure the RAID is synched via checking: cat /proc/mdstat # Now remove one partition: mdadm /dev/md0 --fail /dev/hdd1 --remove /dev/hdd1 # Delete partition, create a new + bigger one and set partition type to fd (Linux raid autodetect): cfdisk /dev/hdd # And re-add the partition: mdadm -a /dev/md0 /dev/hdd1 # Make sure the RAID is synched via checking: cat /proc/mdstat # Repeat the steps for all other disks/partitions as well: mdadm /dev/md0 --fail /dev/hdb1 --remove /dev/hdb1 cfdisk /dev/hdb mdadm -a /dev/md0 /dev/hdb1 cat /proc/mdstat mdadm /dev/md0 --fail /dev/hda1 --remove /dev/hda1 cfdisk /dev/hda mdadm -a /dev/md0 /dev/hda1 cat /proc/mdstat
Now we are ready to resize the RAID5 system online(!) [see ‘man mdadm’ for details]:
root@grml ~ # mdadm --detail /dev/md0 | grep -e 'Array Size' -e 'Device Size' Array Size : 390912 (381.81 MiB 400.29 MB) Device Size : 195456 (190.91 MiB 200.15 MB) root@grml ~ # mdadm --grow /dev/md0 -z max root@grml ~ # mdadm --detail /dev/md0 | grep -e "Array Size" -e 'Device Size' Array Size : 976512 (953.79 MiB 999.95 MB) Device Size : 488256 (476.89 MiB 499.97 MB)
Last step – resize the filesystem (online again):
root@grml ~ # df -h | grep test /dev/md0 370M 15M 336M 5% /mnt/test root@grml ~ # resize2fs /dev/md0 resize2fs 1.39 (29-May-2006) Filesystem at /dev/md0 is mounted on /mnt/test; on-line resizing required Performing an on-line resize of /dev/md0 to 976512 (1k) blocks. The filesystem on /dev/md0 is now 976512 blocks long. root@grml ~ # df -h | grep test /dev/md0 924M 16M 861M 2% /mnt/test
Let’s check whether the testfile is still ok:
root@grml ~ # md5sum -c md5sum /mnt/test/dd: OK
No downtime, no data lost. Excellent. :-) Thanks to Lars Schimmer for the idea and his basic instructions!
September 10th, 2006 at 15:58
very cool.
is it also possible to increase the number of devices of the raid after setting it up, or does one have to comine this with LVM for that purpose?
/thomas
September 10th, 2006 at 22:19
@thomas: yes, adding another device into the RAID5 setup is possible (even without LVM) via running something like:
mdadm /dev/md0 -a /dev/hde1
mdadm –grow /dev/md0 –raid-devices=4
regards,
-mika-