11

I currently have a Btrfs partition with four devices: three 3 TB drives and a 4 TB drive. Data and metadata are RAID 10, so I have 6 TB of usable space, which is almost full. I'd anticipated that RAID 5 support in Btrfs would be mature by the time my storage filled up, but apparently it's not a priority.

My question is: is there a reason to prefer RAID 10 over RAID 1? I know real RAID 1 on my current hardware should give me 3 TB of usable space with 4 copies of each block, but Btrfs apparently does not behave this way. From the Btrfs FAQ:

btrfs combines all the devices into a storage pool first, and then duplicates the chunks as file data is created. RAID-1 is defined currently as "2 copies of all the data on different devices". This differs from MD-RAID and dmraid, in that those make exactly n copies for n devices. In a btrfs RAID-1 on three 1 TB devices we get 1.5 TB of usable data. Because each block is only copied to 2 devices, writing a given block only requires exactly 2 devices to be written to; reading can be made from only one.

And from Jens Erat on Stack Overflow:

Btrfs distributes the data (and its RAID 1 copies) block-wise, thus deals very well with hard disks of different size. You will receive the sum of all hard disks, divided by two – and do not need to think on how to put them together in similar sized pairs.

If more than one disk fails, you're always in danger of losing data: RAID 1 cannot deal with losing two disks at the same time. In your example given above, if the wrong two disks die, you always lose data.

Does this mean that rebalancing from RAID 10 to RAID 1 will give me an extra 500 GB of data (6.5 TB rather than 6 TB) due to the 4 TB drive? And is there any reason for me to stick with RAID 10?

Community
  • 1
Mikkel
  • 905

2 Answers2

7

Yes, you get an extra 500 GB. Note that determining available space in btrfs remains elusive. Also: have a look at the btrfs disk usage calculator.

On your second question: You may lose some performance on your array. Naturally, your data is equally safe on both RAID configurations. When considering performance you can perhaps have a look at these benchmarks: kernel.org, phoronix.com.

Have you perhaps already tried converting to RAID 1? If so: what are your findings?

Mikkel
  • 905
Laura
  • 116
0

Don't use btrfs raid 5 or 6 the implementation is full of bugs, run zfs if you want that.

With raid10 you'll gain speed.

The optimal thing to do is make a 3TB partition on the 4TB drive (make sure all 3 are exactly the same size), combine those in to a RAID10 which will have 6 TB of free space. Use the extra 1 TB on the 4 TB drive for /boot (this has to be vfat anyway if you want to use efistub and avoid grub or any other boot loader which will make your boot a bit faster), swap (you can use a pretty large one, the optimal would be a small zram swap (4 gb works fine with 32 gb ram, 2 gb should be fine with 16 gb, if you have less than 16 idk if it's worth it and you should probably upgrade anyway) that's used first (set the priority to the highest) using zstd and as many compression streams as you have cores/threads... a ssd for / (everything except large media files) and a slightly larger swap with a lower priority (the higher the priority the sooner the swap gets used so the zram one should be highest) on it (8 GB for a 500 GB ssd, 16 for 1 TB, if you have a ssd over 1 TB more than 16 GB swap is probably pointless since if you are swapping that often to it you need more ram and if you have a ssd smaller than 500 GB, consider getting a 500 or 1 TB model since even the best sata ssds are extremely cheap now) and an even larger one on the hdd with the lowest priority (twice as big as the ssd one? Doesn't really matter since if it's getting used often you need more ram... and lowest priority), this still leaves enough space on that drive for another large partition (maybe something for vms or for dual boot with Windows or whatever). Also encrypt your swaps (other than the zram one, since any data on that one will be unrecoverable withing seconds unless someone pours liquid nitrogen on the ram before pulling the power) and / (if you want trim to work on the ssd you need to enable it in crypttab, security is slightly lower since an attacker can see which cells are empty but performance is better).

orange_juice6000
  • 183