The big names mostly can’t yet, but some lesser-known Linux distributions offer the ability to undo updates and recover from damage, even automatically.
With a new version of Linux Mint available, a lot of people – especially less-technical types – are about to upgrade their PCs. As with almost all major upgrades, inevitably some some stuff won’t work with the new version. Going back is difficult, although Mint does have a solution, and so do some smaller distros.
Linux Mint includes the Timeshift utility, originally developed by Teejeetech, but that doesn’t make it invulnerable. No operating system is, as amply demonstrated recently by CrowdStrike. If installed on a file system that supports copy-on-write (COW) snapshots, Timeshift can use that, but if you chose some other disk format, it can also back up your system files using rsync to another drive.
For now, most Linux file systems can’t handle snapshotting, including the default ext4 or its predecessors, or XFS, or the now-vanishing ReiserFS. Several existing file systems that work on Linux do include snapshot support, including bcachefs, ZFS, and Btrfs. However, there are solid reasons why most distros don’t include the functionality.
For instance, while OpenZFS does support snapshots, and several distributions include ZFS, including TrueNAS Scale, Ubuntu, Proxmox, Void Linux, and NixOS, the ZFS code can’t be compiled into the Linux kernel due to an incompatible software license. Thus, from appropriate levels of caution, OpenZFS isn’t part of either SUSE’s or Red Hat’s enterprise offerings. Their compliance with multiple legal requirements and certifications is a big part of their corporate appeal. (For the curious, both companies maintain lists of their certifications. Red Hat has a 16-section list and SUSE a respectable dozen sections. Some companies will pay good money for this stuff.)
Red Hat removed Btrfs from RHEL six years ago, and is working on its own, all-GPL combined volume management system and file system called Stratis. Since Btrfs was invented there, Oracle had a simpler route available for its RHEL-compatible distro: Oracle Linux offers a kernel with Btrfs.
Bcachefs, meanwhile, is relatively new and immature. It only appeared at the beginning of this year in kernel 6.7, meaning that it’s only been in three releases of the kernel so far. It’s a little bit too new for anyone else to support it just yet.
So, for now, Btrfs is the most mature file system in the Linux kernel that can do snapshots. This is why SUSE’s Snapper uses Btrfs by default, although it also supports LVM, and can be configured to use that instead.
Btrfs + Timeshift saved my broken Linux Mint
I love btrfs+snapper. I have automatic snapshots taken before and after every
apt install, so if anything ever goes belly-up, no problem, I just roll back.It’s a little weird sometimes when I’m running out of disk space, so I delete some big downloads and…I get no disk space back, because those files still exist in old snapshots! I suspect there’s some way to finetune it to ignore certain directories (like ~/Downloads) in snapshots but I haven’t taken the time to dig into it. Anyway, it’s not a huge problem because the automatic snapshots are limited to a certain number, so they eventually get bumped out (or I can delete them manually if needed).
I haven’t tried bcachefs yet. Perhaps on my next build.
From what I understand, you would need to move your Downloads-folder to a separate subvolume, as btrfs always does a snapshot of a whole subvolume.
Here on openSUSE, the default setup has the whole /home-folder on a separate subvolume, though. Basically the thought is that, if you need to roll back your OS, you don’t want to also roll back the changes you made to your documents and whatnot.
Also, you should make proper backups of your home-directory anyways, although I guess, you could separately create snapshots of your home-directory, if you want them anyways.
Ah, that makes sense! I probably should have split my /home off to its own subvolume. I’ll add that my list of things to think about next time I hop distros or rebuild (which I’m considering once again, because I have Plasma envy).
And yes, snapshots should NOT be treated as backups. A real backup saves your butt if your drive dies, while a snapshot goes down with that ship. I should really set up a better backup system, but for now I just periodically use Borg to back up to an external HD, and then copy that into an encrypted cloud drive.
If you care about privacy you should encrypt the files first and then upload it to the cloud
Yep, totally agree. Borg supports encryption, so I have the backup encrypted locally, independently of my end-to-end encrypted cloud service. I guess I could store it on a non-e2ee cloud drive and save a little money, but I prefer to keep it simple with an “E2EE or bust” mentality, even for data that’s already encrypted.
For anyone who finds Borg intimidating, take a look at Vorta, an easy-to-use GUI frontend: https://github.com/borgbase/vorta
Yeah my biggest problem with snapper was the disk space.
Also it just… Isn’t super well documented? At least it felt like trying to use it on Fedora was not working as well as it had on opensuse.
I eventually found it to be more trouble than it was worth…
Snapper was too unintuitive for me to figure out and I didn’t want to dig through tutorials and potentially fuck something up. Timeshift has the benefit of being pretty streamlined. It is intuitive and easy to understand even on a first use. The only thing I once had issues was that the background service didn’t start, so it didn’t do auto backups. But that was easy to look up and basically just one command necessary.
Snapper has been wonderful, and saved me several times. I only keep root+home on btrfs and have a separate, larger data partition on ext4 where larger data and nonessential programs end up living.
Why use separate partitions over subvolumes within btrfs?
Mostly because it’s on a separate drive and I hadn’t used btrfs before. Ext4 is ubiquitous and a lot simpler. Besides, I don’t really need the extra features of btrfs for my data volume.