On Fri, 5 Jun 2020 at 12:19, Sven-Hendrik Haase via arch-devops < arch-devops@lists.archlinux.org> wrote:

I'll reiterate the suggestion I made before: Let's get an SX62 and never worry about the archive space again. It's 10€ more. If we want to get it fast, we'll have to order it from the Helsinki location as we can expect long waiting times right now in their German locations (up to 10 workdays for the SX62 according to them). With your AX51 suggestion, we'll have an effective 8TB (RAID1) for 54€/m (Helsinki location) but with the SX62 we'll have an effective 30TB (RAID5) for 64€/m (Helsinki location). It seems like a much better bang for the buck to me.

A 30tb RAID5 is terrifying to my mind.

On 05/06/2020 05:24, Sven-Hendrik Haase via arch-devops wrote:

On Fri, 5 Jun 2020 at 05:14, Phillip Smith via arch-devops mailto:arch-devops@lists.archlinux.org> wrote:

On Fri, 5 Jun 2020 at 12:19, Sven-Hendrik Haase via arch-devops mailto:arch-devops@lists.archlinux.org> wrote:

I'll reiterate the suggestion I made before: Let's get an SX62 and never worry about the archive space again. It's 10€ more. If we want to get it fast, we'll have to order it from the Helsinki location as we can expect long waiting times right now in their German locations (up to 10 workdays for the SX62 according to them). With your AX51 suggestion, we'll have an effective 8TB (RAID1) for 54€/m (Helsinki location) but with the SX62 we'll have an effective 30TB (RAID5) for 64€/m (Helsinki location). It seems like a much better bang for the buck to me.

A 30tb RAID5 is terrifying to my mind.

We can make it RAID6 but we'll have backups anyway.

Softraid or btrfs raid? I guess Phillipe's concern is the downtime when replacing a faulty disk and rebuilding the array? And since this will host dbscripts it's critical infrastructure.

On Fri, 2020-06-05 at 13:13 +1000, Phillip Smith via arch-devops wrote:

On Fri, 5 Jun 2020 at 12:19, Sven-Hendrik Haase via arch-devops < arch-devops@lists.archlinux.org> wrote:

...

I'll reiterate the suggestion I made before: Let's get an SX62 and never worry about the archive space again. It's 10€ more. If we want to get it fast, we'll have to order it from the Helsinki location as we can expect long waiting times right now in their German locations (up to 10 workdays for the SX62 according to them). With your AX51 suggestion, we'll have an effective 8TB (RAID1) for 54€/m (Helsinki location) but with the SX62 we'll have an effective 30TB (RAID5) for 64€/m (Helsinki location). It seems like a much better bang for the buck to me.

A 30tb RAID5 is terrifying to my mind.

Please relax, I've a RAID5 of 6x8TB since years. # df --si|grep home /dev/md0 ext4 40T 37T 3,4T 92% /home # cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md0 : active raid5 sdf[7] sdb[10] sda[11] sdc[9] sdd[8] sde[6] 39070126080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [6/6] [UUUUUU] Sébastien "Seblu" Luttringer

On Sun, 2020-06-07 at 19:11 +1000, Phillip Smith wrote:

On Fri, 5 Jun 2020, 14:06 Sébastien Luttringer, wrote:

...

Please relax, I've a RAID5 of 6x8TB since years.

Have you had to replace a disk and do a rebuild? Yes, unfortunately it's happen. Most of time it's only bad sectors one drive at a time.IIRC the rebuild time was around 20h, depending on the pressure on the RAID.I've a timer with a full mdadm check every month, which scan the whole array. The most soliciting situation is when I upgrade the disks; I have to run 6 rebuild in a row before reshaping. It's obviously worse with consumer drives compared to enterprise drives - I'm not sure what Hetzner provision though. Choosing your disk set and a controller chipset are key decisions for sure and depends a lot of what you want to do with your array, but there is a lot of marketing in the last years around differences between disks which is sometimes only minor firmware tweaking or locking (like scterc, which is useful for RAID arrays) . On the softraid vs btrfs raid question, my feedback is better with md.I had several issues with btrfs raid over years[1].Note that btrfs raid56 is not recommended for metadata and there is still a write hole issue[2].That said, I've a 6x6TB btrfs RAID5 running well since 1 years and the last disk replacement was smooth. Regards, [1] https://lore.kernel.org/linux-btrfs/?q=seblu[2] https://btrfs.wiki.kernel.org/index.php/RAID56

Sébastien "Seblu" Luttringer