A bit of this, a bit of that

My RAID 5 array on a CentOS 5.2 machine just crashed due to a faulty disk so I had to rebuild the array manually. I first thought it would have been the worse time in my life as I do not like too much messing around with my hardware but finally it went pretty smoothly.
So here is how I did to rebuild the RAID, just follow those steps…

First of all you have to look at the problem on md2:

[root@bacchus ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 hdb1[1] hda1[0]
      152512 blocks [2/2] [UU]
md2 : active raid5 sda1[0] sdb1[1]
      625137152 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
md1 : active raid1 hdb2[1] hda2[0]
      117065536 blocks [2/2] [UU]

The faulty device was located on hdc, and on the partition hdc1. So first of all, you have to recreate the partition with the Linux raid autodetect type and write the changes to this partition:

[root@bacchus ~]# fdisk /dev/sdc
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-38913, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-38913, default 38913):
Using default value 38913

Command (m for help): p

Disk /dev/sdc: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       38913   312568641   83  Linux

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): p

Disk /dev/sdc: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       38913   312568641   fd  Linux raid autodetect
Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@bacchus ~]#

Once the partition is created, you have to add the partition to the RAID array with the following command line:

mdadm /dev/md2 -a /dev/sdc1

Once this is done, you just have to monitor the rebuild from time to time with this command:

[gael@bacchus ~]$ cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 hdb1[1] hda1[0]
      152512 blocks [2/2] [UU]

md2 : active raid5 sdc1[3] sda1[0] sdb1[1]
      625137152 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
      [========>............]  recovery = 43.9% (137368104/312568576) finish=85.7min speed=34061K/sec

md1 : active raid1 hdb2[1] hda2[0]
      117065536 blocks [2/2] [UU]

unused devices:

Et voila!, now you just have to wait for you RAID partition to be rebuilt and then you will be an happy bunny