I am using duplicati and thinking of switching to Borg. What do you use and why?
I am old school. I just use GNU Tar with the Pax format and multiple external detachable encypted hard drives. Reason is it is simple and a well known tool that is very common with a standard archive format.
I’m curious - how much data are you backing up with that method and how frequently are you doing your backups? Doesn’t sound like it would scale well, but I’m also wondering if maybe this is perfect and I’ve justbeenoverthinking it.
There is not a size limit. Lot of these other methods actually use GNU Tar behind the scenes anyway. More then that GNU tar has been used for decades for this purpose. Pull out any Unix book from 2 decades ago and you will see “tar”, “cpio”, and “dump/restore” as the way. The new tool out there is Pax and in fact GNU Tar supports the new “pax” format. Moreover GNU Tar with Pax format can backup almost full disk structure including hard links, ACLs, and extended attributes which a lot of tools do not do. It is still useful to archive some things at a lower level like your partition table, and boot blocks of course. You also have to decide what run-level (such as rescue) you want to archive in, and/or what services you should stop, or provide separate to file system dumps for depending on your system. Databases, and things like ecryptfs take some special thought (thought it does for any tool). It is also good to do test restores to verify your disaster plan.
I use tar on many systems. My workstation is about 1TB of data. Backup is about 11 hours though I think it could be faster if I disabled compression (I currently use the standard gzip compression which is not optimal). I think the process is CPU bound by the compression at the moment. Going to uncompressed or using parallel gzip at level 2 is probably the fastest you can do and should really speed things up by 4X or more. I have played with this some for my wife and her raw backup is a lot faster now. My wife uses USB 3 external drives specifically plugged into USB 3 ports (the one with the SS symbol and the blue interior), and with a USB 3 related cable. I use 6TB naked SATA drives I insert into a hot mount enclosure and store in storage boxes. My backup system can theoretically do incrementals too, but it has some issues since I have moved to BTRFS so I do not use that at the moment. Did always use before. I have an idea how to fix, but need to debug and test incrementals now.
How often: I backup monthly. When my incrementals were working I use to do it weekly or whenever I got nervous. Other option for the BTRFS file systems would be to use their native backup tools. Not sure though, I like to use generic stuff. Lot to be said for generic.
Big downside of tar is the mind numbing man page. Getting the options correct takes some real thought. You also have to be comfortable with the shell and Bash scripting. Big upside you can customize exactly what you want.
tar dates all the way back to the 70s.
Yes, I actually did not know how far back, thanks. Wikipedia seems to say 1979. I know my system admin book dated 1992 talks about it and it was common then. I think my brother use to use it in the early 1980s for his job and maybe I did too a few times. Wikipedia says GNU Tar is newer and traces back to 1987. The formats have changed some and there are several. The PAX format is much newer which I think was standardized in 2001 but GNU Tar would have taken time to implement it. I do not know that date.
People seem to forget that tar worked well back then and still does.
I had the chance to play with late 70s Unix for a bit a few years ago. (Hardware on loan from a museum.) VERY minimal, but still recognizable. (Well, my Unix reflexes are old - I started in the mid 80s.)
Interesting. About then I was using a VAX. Somehow I spend most of my time on other stuff until I switched to Linux around 2000.
My first Unix was 4.3BSD on a VAX-11/750. (There was another 11/750 running VMS, but I didn’t like that nearly as much.)
I am using similarly “dumb” back-up system. I’ve two external USB HDDs, to which I copy my home folder every 4 to 6 months. The back-up folder currently has circa 250 GiB, but I don’t use any compression and I also probably do not have to back up my Steam library multiple times.
Yes, it doesn’t scale very well, but at the same time, I do not need to hoard 5 year old data. Yes, I should have an off-site back-up, but if my house burns down, I have bigger problems than losing my old photos.
I use my own scripts with
rsyncetc, I don’t back up my OS itself since I have installing it automated with scripts as well. I just back up specific things I need with my scripts.automated with scripts
would you like to share those or do you have references for creating such scripts? this is on my to do list since years but I always struggle where to begin with.
They’re very personalized to my setup, so they’re not particularly useful in a general sense - I’d recommend something more like using this guide which seems to be pretty good: https://jumpcloud.com/blog/how-to-use-rsync-remote-backup-linux-system
Learning bash has been great for me, it’s helped a ton being able to automate so many different things even just like installing and configuring specific applications to work the way I want, etc
I think a script to manually run for manual backups plus a different script to run for automatic backups scheduled via cronjob is a great way to go.
I’ve used Borg for years now. It’s been rock solid. I test my backups regularly and have done several actual recoveries. I trust it with my data, which is the best thing I can say about backup software.
I use btrfs snapshots and btrbk
btrfs is a great filesystem and btrbk complements it easily. Switching between snapshots is also really easy if something goes wrong and you need to restore.
Archwiki docs for btrfs: https://wiki.archlinux.org/title/Btrfs#Incremental_backup_to_external_drive
Of course you’d still want a remote location to backup to. You can use an encrypted volume with cloud storage. So google drive, etc all work.
Thanks. Heard a lot about it. Will chack it out.
This is what I do. Btrfs snapshots and use send/receive with my NAS.
Oh interesting! I might take a look at btrbk
This is the way !
I’ve used borg for a while and like it a lot. I would say your best option for pure linux is borg+borgmatic/vorta just because borg is battle-tested.
If you run any other OSs and don’t mind a relative newcomer, I’ve found kopia to be easy to recommend to my windows friends. At this point kopia has been around long enough (~4 years of actual beta) that I think it’s safe to trust its integrity with personal data. It has all the important features from borg in a cross-platform solution, so it’s also a viable alternative for borg on linux if you don’t like borg’s frontends for whatever reason.
I’ve tried alternatives but I’ve stuck with LuckyBackup even though there have not been any updates for a while:
- It’s rsync based - which is updated
- It has masses of GUI options including various include/exclude options, pre- and post-commands, etc.
- It’s simple - I can browse inside the backed files and see what is going on, or just restore back one or two files.
- It updates cron itself.
There is no such thing as the objectively best solution. Each tool has advantages and disadvantages. And every user has different preferences and requirements.
Personally, I am using Borg for years. And I have had to restore data several times, which has worked every time.
In addition to Borg, you can also look at Borgmatic. This wrapper extends the functionality and makes some things easier.
And if you want to use a graphical user interface, you can have a look at Vorta or Pika.
Agree. Should say ‘best for you’. Cool thanks. I know of Vorta which I intended of using. Gonna read up on the other ones.
I like pikabackup it’s based on borg
Yeah, this is what I’ve found to be the best option. The encryption and deduplication is great.
I’ve been using restic. It has built-in dedup & encryption and supports both local and remote storage. I’m using it to back up to a local restic-server (pointing to a USB drive) and Backblaze B2.
Restores for single or small sets of files is easy: restic -r $REPO mount /mnt Then browse through the filesystem view of your snapshots and copy just like any other filesystem.
- Btrfs for local system backups based on snapshots
- Photoprism for photos
- Syncthing for other media
You will reconsider calling strategy a backup should the filesystem get corrupted for whatever reason.
I’ve tested my full system backup restore once with btrfs. Worked out fine.
Maybe Photoprism isn’t a backup strategy, but Syncthing for sure is, because you can have multiple backup units in it.
I’m additionally use software RAID on one of devices, that receives Syncthing backups.
Multiple. Locally I have Timeshift doing btrfs snapshots every so often. This is mostly to roll back to a snapshot if something breaks. I’ve never had to use it (and probably should).
I use Pika backup every once in a while for a local backup to an external drive. Mostly because it’s easy to restore quickly.
I have duplicacy doing backups to a cloud provider. I used to use duplicati for this, and it was fine - although I didn’t like that it seems to be forever in beta. I like that duplicacy can do deduplication between backups of different machines which most other solutions I’ve seen cannot. I like its selection of cloud providers vs Borg/Vorta and some others.
I only have backups of my important files, encrypted and synced through rclone.
ZFS snapshots and Borg(matic).
I don’t have backups. :/
And I will regret it some day.
I use github for code so that’s backed up though.
There are two kinds of people.
Those who make backups and those who will.automated/networked backups like people are talking about here are great, but even just an external SSD and the nautilus copy function will give you at least some insurance.
I started using Timeshift when it was included with a distro I was using and haven’t had reason to shift away from it. Have already used it once to do a full restore.
