What backup solutions do you use? A story of a nearly failed backup

[Stal]

Hello everyone,

I was wondering about your backup solutions and strategies, and share some of my experience.

Zorin comes preinstalled with Deja Dup as the backup solution. I've personally used this for several years, with success, until recently.

For those that don't know it Déjà Dup Backups :

Protect yourself from data loss

Déjà Dup is a simple backup tool. It hides the complexity of backing up the Right Way (encrypted, off-site, and regular) and uses duplicity as the backend.

• Support for local, remote, or cloud backup locations such as Google Drive
• Securely encrypts and compresses your data
• Incrementally backs up, letting you restore from any particular backup
• Schedules regular backups
• Integrates well into your GNOME desktop

Déjà Dup focuses on ease of use and recovering from personal, accidental data loss. If you need a full system backup or an archival program, you may prefer other backup apps.

Occasionally, I distrohop. When I do, I install Deja Dup, restore from backup and happily get underway. After restoration I setup Deja Dup to backup every other day.

I recently did this when I tried the Fedora KDE Spin. The restoration worked fine, but when it tried to automatically backup, it would just get stuck at starting to backup. I didn't pay much attention, I was just trying Fedora with KDE, my backups were recent and I didn't add any new meaningful data.

I can't even remember why, but Fedora wasn't working out for me and installed Kubuntu. Again I installed and fired up Deja Dup. This time there was... nothing. Deja Dup could not find any backup. I opened the file explorer to navigate to the backup folder and it was empty. Not a single file, just a 0 byte empty folder. I was shocked.

I tried file recovery tools and managed to recover many gpg encrypted archives. Deja Dup could only make sense of the ones from October 2022 however, and anything since could not be recovered. I tried to manually decrypt the more recent files, but they were from incremental backups and it turned into a mess.

Fortunately I also had a cloud backup/sync going on. It doesn't sync the largest files nor my volatile short-term directories because of bandwidth and storage limitations. Still, after more than 30 hours to download 130 GB of files, I managed to finally get most data back on my PC. It annoyed me that I lost some things, but at least my online storage covered the most critical files.

From now on I'll definitely stick to the 3-2-1 backup rule. I only managed 2-2-1 so until now because I couldn't find the time to clear out another hard-drive to keep another copy on. I've learned my lesson...

TL:DR: My advice to anyone reading this who hasn't done so yet: make backups, at least three, using at least two different methods, and keep at least one separate from the others.

Now as for my question for who's still with me after that wall of text: Which backup software do you use? Deja Dup served me well for years, but as you might have guessed I'm very interested in an additional option.

Note: @LoveZorin mention PikaBackup recently, which builds on BorgBackup rather than Duplicity/rsync. So like LoveZorin I'm curious about people's experience with its reliability.

Thank for reading and commenting.

[swarfendor437]

Once you have installed your OS of choice and have removed programs you don't need and installed the ones you do, use an External drive and use Rescuezilla. If you want to ensure you don't want to risk losing data do NOT use encryption! In terms of creating Snapshots then use Timeshift and save to your /home partition. You can also use an external drive, create a new folder stating OS Name and date, then open your /home folder and if you can't see hidden files (they start with a period, e.g., '.mozilla') press Ctrl+ H then select all contents of your /home folder using Ctrl+ A then Ctrl+ C to copy to clipboard then open newly created folder in external drive and press Ctrl+ V to paste into the folder. Many years ago I purchased SuSE Linux 9.3 Professional. During install it asks if you want to activate encryption, and warned if you chose this option you could risk losing your data. So having read that I never use encryption. At work my Manager wanted her Finance Officer's data to be secure. This was Windows XP and I used encryption for sensitive data. 2 days later, the machine fell over and the encrypted data was inaccessible. Fortunately not much data was lost as the Finance Officer had backed up their data. Encryption uses OS ID, so if a system needs reinstalling, the machine ID changes which the encrypted partition relies on, so not possible to recover. I always felt Deja Dup was a poor option compared to other options, so have never used it.

[zenzen]

All of my documents, pictures and other important files are stored on separate attached hard drives. This way, all I need to restore are my installed programs and configuration files.
In that regard, distributions like NixOS and GuixOS have caught my attention as of lately. The idea of an immutable and reproducible OS is very interesting.

For now though, I've been putting together a handy Ansible playbook that will do this for me automatically so that I can replicate my setup on any machine, running any distribution. This is obviously very comfortable but it also helps me keep track of what I installed or downloaded, how it's configured, etc.

I've never used a snapshot tool before, other than Snapper which comes installed and pre-configured in OpenSUSE Tumbleweed. The process is completely transparent and has worked flawlessly the few times I used it. I can imagine myself getting used to this so I will give Timeshift a try.

[Stal]

@swarfendor437

During install it asks if you want to activate encryption, and warned if you chose this option you could risk losing your data. So having read that I never use encryption.

This is an interesting point of view... it's completely new to me and I wonder what the original rationale behind this was. Ever since switching to Linux, I've generally always encrypted my data because it's been so easy and reliable. Since I have backups I've never had the encryption lead to data loss on broken installations. I've never even attempted to recover encrypted data after a re-installation and just thought of it as gone, so I never considered encryption as a possible problem.

[CC7]

Timeshift with 3 daily and 2 weekly snapshots. In my 8 months with Linux I've only had to use it twice.

I use VMs as of recent when experimenting with things so now I really only have Timeshift for emergencies.

[Stal]

Do you use Timeshift to backup personal files? I thought it was intended for system restore? Or is that just Linux Mint’s interpretation?

[swarfendor437]

By default it excludes /home but you can turn this option on. You also need to ensure that you manually partition with 500 Mb for EFI, 80 Gb Primary for ’/’, an extended partition at the end of which you have 'swap area' which should be double the amount of your physical RAM and everything in front of the swap area as /home. You could also have /home on a separate drive altogether.

[zabadabadoo]

I sort of remember a user having a problem restoring /Home from DejaDup backup after changing distro.

Since then I have been using DejaDup for regular /Home backup, but also copy (drag) the entire contents of /Home to a dedicated folder on external drive.
I believe @Aravision does this, but compresses the backup file, I don't bother with compression as size is not great.

Using DejaDup, I always do a fresh (not incremental) backup to a dedicated external drive folder, without restore password, to keep it simple as possible. A fresh backup takes longer but is stand-alone and simple.

[CC7]

everything

[Storm]

What's backup? Do as me living on the edge and curse when it goes wrong

The only thing I backup is my personal files and stuff I make and I do it manually.

[Stal]

Sounds like you're backing up to me. At least from my point of view. I don't store anything beyond personal files either.

I've never restored systems and settings, because when I reinstall I generally like a fresh experience. I guess I'm stuck in that habit after many years of using Windows where a fresh install always ran so much faster, until it got bogged down again.

The time I spent configuring my system is also negligible compared to the time it takes to restore my personal files and download and install updates. I've never gotten to the point of really fine-tuning my system, also because I like different DEs.

[LoveZorin]

Hello @Stal,

Thank you for mentioning my post about Pika Backup tool.

So, I have been using Pika for the last 2 months. I am definitively switching from Deja Dup to Pika. The interface and browser tool management for backups in Pika are really good. However, I have not seen any comments from users with more experience yet.

Have you tried the tool?

What do you think about it?

[Stal]

I have not really used it yet, but I’m thinking of using it side by side with Dejadup. My external drives have six times more storage capacity than my internal SSD, so aside from requiring more time I should be able to maintain another backup.

I’ll try using it for a month or so for regular incremental backups and try testing restoring from backup: A backup is no backup unless you know it works.

[Stal]

For those curious: I’ve been making regular backups with both Deja Dup and Pika backup in parallel and I’ve successfully restored from both.

I feel the initial backup of Pika took longer, but after that I did not notice any difference. These backups were unencrypted for both, I don’t know if encryption would have affected this.

In restoring, Deja Dup makes it easier to just restore everything to original locations, if for example you have created a fresh install. On the other hand, Pika lets you easily browse files in a file manager and you have much more granular control of restoring. It’s easy to restore an individual file in case of corruption or something.

I think both have there merit.

In general Pika feels more transparent in use somehow, but Deja Dup has much more widespread availability. So again something to say for each.

If you care about looks, Pika is a GTK4 adwaita app, and it’ll look out of place on themed or non gnome desktops. In general Deja Dup with the versions that distributions have in their repositories looks more native.

It’s difficult to recommend one over the other in general and I think it comes down to personal preference. I have yet to test the (very) long term stability of archives with incremental updates. But at this moment I feel either is good and better than having to manually copy and paste files to backup. I also prefer the experience of either compared to something like grsync.