Best disk config approach for data integrity

[atwillia]

Hi,

I'm about to install Zorin 17 but wanted to first get some advice on how best to configure my storage. I have two 4TB NVME drives in my computer. My primary objective/use case is to use my computer as my daily driver for everyday productivity tasks and photography/home movie editing and storage. While I back up my data regularly, I am keen to avoid losing my family photos and movies over time to bit-rot. I do have a Synology NAS where I store everything and run BTRFS with checksum scrubbing but wanted to try and do the same on my desktop.

Is there a way to do that using LVM and BTRFS? Or, should I use ZFS? In either case, I'm pretty sure I will need to mirror the drives to give the filesystem a way to self-heal. Is it possible to install Zorin on a RAID1 mirror? Or, should I add in a 3rd SSD, load the OS there and then use the two 4TB drives for data storage under ZFS or BTRFS? This is actually how I had my Windows 11 box configured - one drive for C: and the two drives mirrored using Storages Spaces and formatted with REFS for D:.

Any advice/guidance will be greatly appreciated!

Thanks!

[swarfendor437]

@Mr_Magoo - I think this is the person best to help. RAID and NVMe is beyond (NV)Me! Have you thought about saving images and video to Blu-Ray DVDs?

[Mr_Magoo]

I wouldn't recommend using ZFS unless you've got at least 3 drives to mirror... the failure of one drive in a 2 drive mirror makes for a pretty precarious situation, as you're completely relying upon that other drive to resilver the new drive you drop in.

And even then, I'd only recommend it for the OS (mainly to combat bitrot, and to speed up spinning-rust drive throughput), not for your personal files.

A good way of making backup copies is to clone a drive, save that .IMG file to a different drive, then compress it with .7z to save space. If you're diligent with that, you don't need a mirrored-drive setup, and you can save several iterations of the backed-up drives so if one compressed backup is corrupt, you can roll back to older ones.

Zeroing the unused space on a drive makes the .7z compression work much better... I can compress three 500 GB drive .IMG files down to a single 8.8 GB .7z file (because, again, I've only got ~1% drive space utilization... ZFS is a CoW (Copy-on-Write) system, so it never writes to the original sectors that the file was on, it finds new, unused space... the problem with that is, there are a ton of sectors with abandoned but non-zero'd data, and that doesn't compress well. Zeroing that free space allows it to compress well).

The OS itself doesn't need a lot of space... I'm running 500 GB drives (3 of them) mirrored on both the ZFS bpool and rpool, and I've still got 99% free space.

So get something small and fast for the OS (100 GB would be more than enough (unless you're installing huge games), mirrored across 3 or more drives, the number of drives depending upon how many separate drive interfaces you've got (it won't speed things up to have, for instance, multiple drives mirrored on the same USB port via a hub), then use your big honking drives as personal file storage and backup of that personal file storage.

Keep your personal files separate from the OS (rather than putting them in the directories provided by the OS... or you can do this)... if you need to bug out due to a natural catastrophe or fire or whatever, you can just grab that external drive containing your personal files (ie: your data is far more valuable than your computer). Format it as NTFS and you can then plug it into any handy computer (Linux, Windows or Mac) to access your personal files, which might come in handy in an apocalyptic situation where computers are scarce.

[atwillia]

Thanks for the quick response and advice! As I only have just the three drives, I’ll probably just stick with the OS on a single drive and then maybe just use LVM to combine the the other two drives and back them as you described.

[Mr_Magoo]

If you've got a gaming rig and NVME or SSD drives, ZFS is ideal... the NVME or SSD drives are already fast, putting them into a ZFS mirrored setup on separate interfaces interleaves data reads and increases the read throughput.

My three spinning-rust drives in a ZFS mirrored setup is of a similar read speed as an SSD, without having to worry about write-wearing.

The write speed on a mirrored setup is going to be the same as a single drive, but you can speed that up by mirroring SSDs on separate interfaces as SLOG drives. They'll write-wear, though, and eventually require replacement. The SLOG drives don't need to be huge... 25 GB would be plenty.