Frequently asked questions -- Debian mdadm ========================================== Also see /usr/share/doc/mdadm/README.recipes.gz 0. What does MD stand for? ~~~~~~~~~~~~~~~~~~~~~~~~~~ MD is an abbreviation for "multiple device" (also often called "multi- disk"). The Linux MD implementation implements various strategies for combining multiple physical devices into single logical ones. The most common use case is commonly known as "Software RAID". Linux supports RAID levels 1, 4, 5, 6, and 10, as well as the "pseudo-redundant" RAID level 0. In addition, the MD implementation covers linear and multipath configurations. Most people refer to MD as RAID. Since the original name of the RAID configuration software is "md"adm, I chose to use MD consistently instead. 1. How do I overwrite ("zero") the superblock? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ mdadm --zero-superblock /dev/mdX Note that this is a destructive operation. It does not actually delete any data, but the device will have lost its "authority". You cannot assemble the array with it anymore, and if you add the device to another array, the synchronisation process *will* *overwrite* all data on the device. Nevertheless, sometimes it is necessary to zero the superblock: - If you are reusing a disk that has been part of an array with an different superblock version and/or location. In this case you zero the superblock before you assemble the array, or add the device to an array. - If you are trying to prevent a device from being recognised as part of an array. Say for instance you are trying to change an array spanning sd[ab]1 to sd[bc]1 (maybe because sda is failing or too slow), then automatic (scan) assembly will still recognise sda1 as a valid device. You can limit the devices to scan with the DEVICE keyword in the configuration file, but this may not be what you want. Instead, zeroing the superblock will (permanently) prevent a device from being considered as part of an array. 2. How do I change the preferred minor of an MD array (RAID)? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ See item 12 in /usr/share/doc/mdadm/README.recipes.gz and read the mdadm manpage (search for 'preferred'). 3. How does mdadm determine which /dev/mdX or /dev/md/X to use? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The logic used by mdadm to determine the device node name in the mdadm --examine output (which is used to generate mdadm.conf) depends on several factors. Here's how mdadm determines it: It first checks the superblock version of a given array (or each array in turn when iterating all of them). Run mdadm --detail /dev/mdX | sed -ne 's,.*Version : ,,p' to determine the superblock version of a running array, or mdadm --examine /dev/sdXY | sed -ne 's,.*Version : ,,p' to determine the superblock version from a component device of an array. Version 0 superblocks (00.90.XX) '''''''''''''''''''''''''''''''' You need to know the preferred minor number stored in the superblock, so run either of mdadm --detail /dev/mdX | sed -ne 's,.*Preferred Minor : ,,p' mdadm --examine /dev/sdXY | sed -ne 's,.*Preferred Minor : ,,p' Let's call the resulting number MINOR. Also see FAQ 1 further up. Given MINOR, mdadm will output /dev/md if the device node /dev/md exists. Otherwise, it outputs /dev/md/ Version 1 superblocks (01.XX.XX) '''''''''''''''''''''''''''''''' Version 1 superblocks actually seem to ignore preferred minors and instead use the value of the name field in the superblock. Unless specified explicitly during creation (-N|--name) the name is determined from the device name used, using the following regexp: 's,/dev/md/?(.*),$1,', thus: /dev/md0 -> 0 /dev/md/0 -> 0 /dev/md_d0 -> _d0 /dev/md/d0 -> d0 /dev/md/name -> name (/dev/name does not seem to work) mdadm will append the name to '/dev/md/', so it will always output device names under the /dev/md/ directory. If you want to change the name, you can do so during assembly: mdadm -A -U name -N newname /dev/mdX /dev/sd[abc]X I know this all sounds inconsistent and upstream has some work to do. We're on it. 4. Which RAID level should I use? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Please read /usr/share/doc/mdadm/RAID5_versus_RAID10.txt.gz . Many people seem to prefer RAID4/5/6 because it makes more efficient use of space. If you have disks of size X, then in order to get 2X of usable space, you need e.g. 3 disks with RAID5, but 4 if you use RAID10 or RAID1+0. This gain in usable space comes at a price: performance; RAID1/10 can be up to four times faster than RAID4/5/6. At the same time, however, RAID4/5/6 provide somewhat better redundancy in the event of two failing disks. In a RAID10 configuration, if one disk is already dead, the RAID can only survive if any of the two disks in the other RAID1 array fails, but not if the second disk in the degraded RADI1 array fails. A RAID6 across four disks can cope with any two disks failing. If you can afford the extra disks (storage *is* cheap these days), I suggest RAID1/10 over RAID4/5/6. If you don't care about performance but need as much space as possible, go with RAID4/5/6, but make sure to have backups. Heck, make sure to have backups whatever you do. Let it be said, however, that I thoroughly regret putting my primary workstation on RAID5. Anything disk-intensive brings the system to its knees; I will have to migrate to RAID10 at one point. 5. What is the difference between RAID1+0 and RAID10? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ RAID1+0 is a form of RAID in which a RAID0 is striped across two RAID1 arrays. To assemble it, you create two RAID1 arrays and then create a RAID0 array with the two md arrays. The Linux kernel provides the RAID10 level to do pretty much exactly the same for you, but with greater flexibility (and somewhat improved performance). While RAID1+0 makes sense with 4 disks, RAID10 can be configured to work with only 3 disks. Also, RAID10 has a little less overhead than RAID1+0, which has data pass the md layer twice. I prefer RAID10 over RAID1+0. 6. (One of) my RAID arrays is busy and cannot be stopped. What gives? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is perfectly normal for mdadm to report the array with the root filesystem to be busy on shutdown. The reason for this is that the root filesystem must be mounted to be able to stop the array (or otherwise /sbin/mdadm does not exist), but to stop the array, the root filesystem cannot be mounted. Catch 22. The kernel actually stops the array just before halting, so it's all well. If mdadm cannot stop other arrays on your system, check that these arrays aren't used anymore. Common causes for busy/locked arrays are: * The array contains a mounted filesystem (check the `mount' output) * The array is used as a swap backend (check /proc/swaps) * The array is used by the device-mapper (check with `dmsetup') * LVM * dm-crypt * EVMS * The array is used by a process (check with `lsof') 7. Should I use RAID0 (or linear)? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ No. 7b. Why not? ~~~~~~~~~~~~ RAID0 has zero redundancy. If you stripe a RAID0 across X disks, you increase the likelyhood of complete loss of the filesystem by a factor of X. The same applies to LVM by the way. If you want/must used LVM or RAID0, put it on RAID1 arrays (RAID10/RAID1+0, or LVM on RAID1). -- martin f. krafft Wed, 23 Sep 2006 10:28:29 +0100 $Id$