I have been lucky. I have had very few critical hard drive failures in my life. But as time goes on rolling those dices will be more risky so.. Lets see how hard is to set up RAID array.

In this small write up I will show how to set up RAID6 array with 4 drives.

Hardware that we will use is:
RaspberryPi 3B+
USB HUB
5x USB memory sticks (Any size will work).

Before we start learn from my failures.

 

  1. Make sure that your USB hub provides enough power for your memory sticks.
  2. USB and/or USB hub is not reliable, in most cases, hardware to hold your RAID array.
  3. On RaspberryPi you may need to play around with different distributions.

The amount of failures while doing this small project was just crazy. Memory sticks stop responding, hub not working correctly and everything being finicky overall. So this setup is not recommended for real data in any shape or form.
But it is best setup to test and learn about RAID because everything and anything that can go wrong will go wrong. So you can learn to rebuild and recover your arrays.

 

Ok lets get started

  1. We will use ubuntu 18.04.3 server for raspberry pi. (ubuntu-18.04.3-preinstalled-server-arm64+raspi3.img). I started with version 19.10 but for some reason RPi hanged couple of seconds after starting to create array.
    Install the os and set it up. Updates, ssh if you like, networking and the whole nine yards.
  2. Insert 4 of the usb drives in to the hub.
  3. Now check that OS sees the drives. You should see devices sda, sdb, etc:
    ubuntu@ubuntu:~$ sudo lsblk
    NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
    sda           8:0    1 28.7G  0 disk 
    └─sda1        8:1    1 13.7G  0 part 
    sdb           8:16   1   29G  0 disk 
    └─sdb1        8:17   1 13.7G  0 part 
    sdc           8:32   1 28.7G  0 disk 
    └─sdc1        8:33   1 13.7G  0 part 
    sdd           8:48   1 14.6G  0 disk 
    └─sdd1        8:49   1 13.7G  0 part 
    sde           8:64   1 28.7G  0 disk 
    └─sde1        8:65   1 13.7G  0 part 
    mmcblk0     179:0    0 29.7G  0 disk 
    ├─mmcblk0p1 179:1    0  256M  0 part /boot/firmware
    └─mmcblk0p2 179:2    0 29.5G  0 part /

     

  4. My drives are already partitioned but lets take a look how to partition drives. All the partitions you include in the array have to be same size. Because one of my drives was smaller (14.6GB) I decided to create 13.7GB partitions on all the drives.
    Run:

    ubuntu@ubuntu:~$ sudo fdisk /dev/sda
    
    
    Welcome to fdisk (util-linux 2.31.1).
    Changes will remain in memory only, until you decide to write them.
    Be careful before using the write command.
    
    
    Command (m for help): n
    Partition type
       p   primary (0 primary, 0 extended, 4 free)
       e   extended (container for logical partitions)
    Select (default p): p
    Partition number (1-4, default 1): 
    First sector (2048-60063743, default 2048): 
    Last sector, +sectors or +size{K,M,G,T,P} (2048-60063743, default 60063743): 28674047
    
    Created a new partition 1 of type 'Linux' and of size 13.7 GiB.
    
    Command (m for help): w
    The partition table has been altered.
    Calling ioctl() to re-read partition table.
    Syncing disks.

    you have to do it to all the drives (/dev/sda, /dev/sdb …) you want to include in the array.

  5. After that you get drives and partitions that look similar to this:
    ubuntu@ubuntu:~$ sudo lsblk
    
    NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
    sda           8:0    1 28.7G  0 disk 
    └─sda1        8:1    1 13.7G  0 part 
    sdb           8:16   1   29G  0 disk 
    └─sdb1        8:17   1 13.7G  0 part 
    sdc           8:32   1 28.7G  0 disk 
    └─sdc1        8:33   1 13.7G  0 part 
    sdd           8:48   1 14.6G  0 disk 
    └─sdd1        8:49   1 13.7G  0 part 
    sde           8:64   1 28.7G  0 disk 
    └─sde1        8:65   1 13.7G  0 part 
    mmcblk0     179:0    0 29.7G  0 disk 
    ├─mmcblk0p1 179:1    0  256M  0 part /boot/firmware
    └─mmcblk0p2 179:2    0 29.5G  0 part /
  6. Command for creating array is quite simple. You need to indicate the name/device of the array (dev/md0) how many drives/devices you will have in the array (–raid-devices=4), raid level (–level=6) and partitions/devices (/dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1)
    depending on your drive letters you may need to change them. If all looks good run:

    ubuntu@ubuntu:~$ sudo mdadm --create --verbose /dev/md0 --level=6 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
    
    mdadm: layout defaults to left-symmetric
    mdadm: layout defaults to left-symmetric
    mdadm: chunk size defaults to 512K
    Continue creating array? 
    Continue creating array? (y/n) y
    mdadm: Defaulting to version 1.2 metadata
    mdadm: array /dev/md0 started.
  7. Now the array is being built. To check the status rund:
    ubuntu@ubuntu:~$ cat /proc/mdstat
    Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
    md0 : active raid6 sdd1[3] sdc1[2] sdb1[1] sda1[0]
          43920384 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/5] [UUUU]
          [>....................]  resync =  0.8% (118532/14640128) finish=82.7min speed=2924K/sec
    
    unused devices: <none>

    Or you can monitor the progress with the following command

    watch -d cat /proc/mdstat

    This will take time and depending on the size and count of drives the waiting can get long.

  8. If build has finished you can check the status:
    ubuntu@ubuntu:/$ sudo mdadm -D /dev/md0
    /dev/md0:
               Version : 1.2
         Creation Time : Mon Dec  9 11:57:54 2019
            Raid Level : raid6
            Array Size : 28653568 (27.33 GiB 29.34 GB)
         Used Dev Size : 14326784 (13.66 GiB 14.67 GB)
          Raid Devices : 4
         Total Devices : 4
           Persistence : Superblock is persistent
    
           Update Time : Mon Dec  9 20:05:29 2019
                 State : clean 
        Active Devices : 4
       Working Devices : 4
        Failed Devices : 0
         Spare Devices : 0
    
                Layout : left-symmetric
            Chunk Size : 512K
    
    Consistency Policy : resync
    
                  Name : ubuntu:0  (local to host ubuntu)
                  UUID : 9970b426:1cd1ba78:7d4fd643:81a4753a
                Events : 17
    
        Number   Major   Minor   RaidDevice State
           0       8        1        0      active sync   /dev/sda1
           1       8       17        1      active sync   /dev/sdb1
           2       8       33        2      active sync   /dev/sdc1
           3       8       49        3      active sync   /dev/sdd1
  9.  Now we need to create a file system on to the new array device (/dev/md0). I’m using ext4 in this example.
    ubuntu@ubuntu:/media$ sudo mkfs.ext4 /dev/md0
    mke2fs 1.44.1 (24-Mar-2018)
    Creating filesystem with 7163392 4k blocks and 1794048 inodes
    Filesystem UUID: b80f7697-902b-46ba-aa81-1190557bb7f0
    Superblock backups stored on blocks: 
      32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
      4096000
    
    Allocating group tables: done                            
    Writing inode tables: done                            
    Creating journal (32768 blocks): done
    Writing superblocks and filesystem accounting information: done
  10. Next we create a mound point for the array. I used my /media folder and used name “raid6” for mount folder. But you are free tho choose.
    sudo mkdir /media/raid6
  11. Lets add the mount point to the fstab. Open fstab file:
    sudo nano /etc/fstab

    and add the following line:

    /dev/md0 /media/raid6/ ext4 defaults 1
    
    
  12. Next lets mount the array with mount command:
    sudo mount -a
  13. Now if you run:
    ubuntu@ubuntu:/$ df -h
    Filesystem      Size  Used Avail Use% Mounted on
    udev            441M     0  441M   0% /dev
    tmpfs            92M  4.7M   87M   6% /run
    /dev/mmcblk0p2   29G  2.2G   26G   8% /
    tmpfs           457M     0  457M   0% /dev/shm
    tmpfs           5.0M     0  5.0M   0% /run/lock
    tmpfs           457M     0  457M   0% /sys/fs/cgroup
    /dev/mmcblk0p1  253M   95M  158M  38% /boot/firmware
    tmpfs            92M     0   92M   0% /run/user/1000
    /dev/md0         27G   45M   26G   1% /media/raid6

    You should see your mounted array.

    Optional
  14. Right now your mound point is with root permissions so if you want to use with non root users run:
    sudo chown -r ubuntu:ubuntu /media/raid6

    And that’s it.