BorgBackup 2.0 supports Rclone – over 70 cloud providers in addition to SSH
borgbackup.readthedocs.ioGet the top HN stories in your inbox every day.
Helmut10001
jszymborski
Or, if you're using a desktop environment and prefer a GUI, Vorta has been treating me well.
mrbigbob
Another fairly good gui for Borg is Pika Backup. https://gitlab.gnome.org/World/pika-backup
mystified5016
I love vorta, I have it on all my machines. It's great for my (non-technical) husband who doesn't want to know anything about Linux.
Then everything gets backed up to my local server which then syncs out to remote storage. It's great.
Can't wait for Borg 2 to hit stable. The transfer command solves so many problems
dudu24
My problem is I learn some tool like this, set it, and then indeed forget it. Then I avoid testing my backups because of the work it takes to un-forget it. Because of this, I'm leaning more and more towards rsync or tools that have GUI frontends.
belthesar
Rather than avoid tools that work well, I would encourage you to adopt solutions that solve your use cases. For instance, if you aren't getting notifications that a backup is running, completing or failing, then all you've set up is a backup job, and not built a BDR process. If you're looking for a tool to solve your entire BDR plan, then you're looking at a commercial solution that bakes in automating restore testing, so on and so forth.
Not considering all the aspects of a BDR process is what leads to this problem. Not the tool.
ohthehugemanate
At a minimum you need backup, regular restore tests, and alerts when backups stop or restore tests fail.
Personally I automate restore testing with cron. I have a script that picks two random files from the filesystem: an old one (which should be in long term storage) and a new one (should be in the most recent backup run, more or less), and tries restoring them both and comparing md5sums to the live file. I like this for two reasons: 1. it's easy to alert when a cronjob fails, and 2. I always have a handy working snippet for restoring from backups when I inevitably forget how to use the tooling.
IMO alerting is the trickiest part of the whole setup. I've never really gotten that down on my own.
Helmut10001
I recently set up email alerting through the syslog agent from Telegraf-Influx-Grafana, where Grafana is used for Email alerting and InfluxDB for filtering for the specific syslogs.
On another VM, I used postfix to email logs after cronjob (failed or passed), which also works great.
amjd
I use ntfy.sh for sending push notifications from scripts and such. It's open source and free (they have paid plans as well now, but I didn't encounter any limitations in the free plan).
Not an endorsement, just a happy user.
mbrumlow
> set and forget for me and after 7 years
Please tell me you verify your backups now and then?
Helmut10001
Borgmatic runs consistency checks [1] once a month on all repositories and archives and I occasionally retrieve older versions for selected files (archives with --verify-data only once a year or whenever I feel the need - there's 9TB of data in the borg repo, which takes a bit to scan). Note though that borg is not my main backup, it is the fallback "3" in the 3-2-1 principle, where my primary data is a ZFS Raidz2 and my primary backup is an offsite ZFS Raidz2 in pull mode. I added borg because I did not want to rely on a single software (ZFS), although this fear was unstained so far.
[1]: https://borgbackup.readthedocs.io/en/stable/usage/check.html
witten
You may also be interested in borgmatic's (beta) "spot" check, which compares your backup archive's files to your source files: https://torsion.org/borgmatic/docs/how-to/deal-with-very-lar...
dewey
This always gets repeated, sounds good and makes sense theoretically but in reality there's no good way to do that and it should be the job of a computer to that.
Restoring one file from the backup, works but what if something else is corrupted?
Restoring the system from the image, works but what if some directory is not in the backup and you don't see that while testing?
l33tman
I think the point is that if your data is valuable enough for you, you can't really trust that option in the backup tool to work - maybe you misunderstood some config option and the test now isn't really run, the tool is broken, or is run only on some of the backup files or dirs etc... or your original config might have missed a folder because it was mounted through some other filesystem (happened to me with Borg actually, and my whole /home/user dir especially wasn't backed up for the first 6 months I ran it :).
Seems to be good to have another tool that you either manually or automatically can setup to run regularly that tries to locate random files from your existing file system in the backups? Something like that.. though that other tool might be broken as well of course... :/
witten
borgmatic's "spot" check (probabilistically) protects against both of those failure modes: https://torsion.org/borgmatic/docs/how-to/deal-with-very-lar...
iphoneisbetter
[flagged]
selcuka
> Please tell me you verify your backups now and then?
Then one can't call it "set and forget", right?
semanticist
Backup testing can be automated. I don't do this for my personal stuff, but at work there's a box that does a restore of our primary DB from backups, loads it into MySQL, and runs some smoke tests to make sure it looks roughly right. A quick script and a cronjob, and backups get tested every night.
I'm sure there's more thorough ways to do this kind of testing, but whatever level of confirmation you need automating it should be viable and then you only have to pay attention if/when something breaks.
rubenbe
Does someone know a good Android client?
ThomasWaldmann
Some people were installing / using borg on android, but guess that isn't suitable for end users, rather for the nerds.
Maybe try SeedVault?
uhartelightning
I syncthing stuff I want off of my phone onto a computer, from which I borg-back-it-up.
delduca
+1 borgmatic
krick
Currently I'm just using bare rclone to backup to my own remote machines, but obviously this isn't very professional solution. Was thinking to add Backblaze B2 as a remote, but I guess using rclone wouldn't be a state-of-the-art solution here. After all, it isn't really a backup tool, is it? It has some built-in encryption, but it's a bit clunky, and I'd think a proper backup tool should automatically divide data into blocks of suitable size (instead of just creating file-per-file - to make it S3/B2 API-friendly), encode whole directories as tar (if needed to preserve links, for example), do deduplication, and whatever else are best practices I have no idea about, but which backup-proficient people probably invented long time ago.
Does anybody have a recommendation?
I briefly looked at restic and duplicati, but surprisingly none are as simple to use as I'd expect a dedicated backup-tool to be (I don't need, and kindda don't want GUI, I'd like all configuration to be stored in a single config-file I can just back-up to a different location like everything else, and re-create on any new machine). More than that, I've read some scary stories about these tools fucking up their indexes so that data turns out to be non-restorable, which sounds insane, since this is something you must be absolutely sure your backup-tool would never do no matter what, because what's even the point of making backups then.
RockRobotRock
>I'd like all configuration to be stored in a single config-file I can just back-up to a different location like everything else, and re-create on any new machine
You might want to look into kopia. It accomplishes the same task as restic, but handles configs in a way you might find more appealing. Further reading: https://news.ycombinator.com/item?id=34154052
Don't even bother with duplicati. I've tried to make it work so many times, but it's just a buggy mess that always fails. It's a shame too, because I really like the interface.
unaindz
I've been using bupstash since trying to do backups on an rpi and finding Borg too slow to be usable. Since then I upgraded to a proper server at home but kept bupstash as I found it to just work better for the most part. Keep in mind there's not been much progress since the last release two years ago and its still tagged as beta by the author. Tbf I think he has a higher quality standard than in other projects that are not tagged as such.
Useful backup tool comparison: https://github.com/deajan/backup-bench
scorpioxy
Whether something is simple or not I'd say depends on the use case. But I found borg to be great. I'd recommend you check it out and go through the quickstart guide in the documentation. It does de-duplication and encryption. It does a lot more but you don't have to use those features if you don't need them. I couple it with borgmatic to implement a backup and disaster recovery procedure that is meant to decrease the risk of data loss. I also use borgbase and they have a good service but using something like B2 with this rclone support would result in a cheaper alternative if you don't need the extra that borgbase provides.
I've been using it for quite a while now both for my personal projects and paid work and have had a good experience with it.
abhinavk
restic + autorestic/resticprofile.
Borg 2 is still beta and Kopia is also there. But it's newer so I am testing it on another redundant backup on the same machine. I have space so why not?
Every once in a while I run integrity check (with data) so I can trust that metadata and data are fine.
Mister_Snuggles
I'm very happy with Restic backing up to BackBlaze B2.
I have a "config file", which is really just a shell script to set up the environment (repository location, etc), run the desired backups, and execute the appropriate prune command to implement my desired retention schedule.
I've been using this setup for years with great success. I've never had to do a full restore, but my experience restoring individual files and directories has been fine.
Do you have any links related to the index corruption issue? I've never encountered it, but obviously a sample size of one isn't very useful.
nickcw
Writing an rclone backend for borg is something I have wanted to do for a long time.
However I found that the backends weren't well abstracted enough in v1 to make that easy.
However for v2 Thomas Waldmann has made a nice abstracted interface and the rclone code ended up being being only <300 lines of Python which only took an afternoon or two to make.
https://github.com/borgbackup/borgstore/blob/master/src/borg...
ThomasWaldmann
Thanks a lot for writing the rclone backend!
scorpioxy
Oh very interesting. This has been a requested feature for a while especially with the rise in popularity and the decreased cost of object storage.
Borg working with object storage was not supported though some people did use it that way. From my understanding, most would duplicate a repo and upload instead of borg directly writing/manipulating it. This could problematic if the original repo was corrupt as now the corruption would be duplicated. So this will make things much easier and allow for a more streamlined workflow. Having the tool support rclone instead of specific services seems like a wise and more future-proof choice to me.
dang
Related. Others?
Borg 2.0 beta (deduplicating backup program with compression and encryption) - https://news.ycombinator.com/item?id=40990425 - July 2024 (1 comment)
Borgctl – borgbackup without bash scripts - https://news.ycombinator.com/item?id=39289656 - Feb 2024 (1 comment)
BorgBackup: Deduplicating archiver with compression and encryption - https://news.ycombinator.com/item?id=34152369 - Dec 2022 (177 comments)
Emborg – Front-End to Borg Backup - https://news.ycombinator.com/item?id=30035308 - Jan 2022 (2 comments)
Deduplicating Archiver with Compression and Encryption - https://news.ycombinator.com/item?id=27939412 - July 2021 (71 comments)
BorgBackup: Deduplicating Archiver - https://news.ycombinator.com/item?id=21642364 - Nov 2019 (103 comments)
Borg – Deduplicated backup with compression and authenticated encryption - https://news.ycombinator.com/item?id=13149759 - Dec 2016 (1 comment)
BorgBackup (short: Borg) is a deduplicating backup program - https://news.ycombinator.com/item?id=11192209 - Feb 2016 (1 comment)
Mister_Snuggles
Does anyone have an up-to-date comparison of Borg vs Restic? Or a compelling reason to switch from Restic to Borg?
I've previously used Borg, but the inability to use anything other than local files or ssh as a backend became a problem for me. I switched to Restic around the time it gained compression support. So for my use-case of backing up various servers to an S3-compatible storage provider, Restic and Borg now seem to be equivalent.
Obviously I don't want to fix what isn't broken, but I'd also like to know what I'm missing out on by using Restic instead of Borg.
ectospheno
I prefer restic simply because I find it easier to understand and use. This means backups actually happen. It also feels less like it is constantly changing. Constant stream of new features isn’t a thing I’ve ever desired in a backup solution.
kornnflake
+1, I'm in a similar situation and be curious too about an up-to-date comparison.
ThomasWaldmann
Comparisons might be interesting, but one needs to be aware that they would be a bit apples to oranges:
- unreleased code that is still in heavy development (borg2, especially the new repository code inside borg2).
- released code (restic) that has practically proven "cloud support" since quite a while.
borg2 is using rclone for the cloud backend, so that part is at least quite proven, but the layers above that in borg2 are all quite fresh and not much optimized / debugged yet.
cstuder
If you're looking for cheap online storage for your backups know this: A Microsoft 365 Single subscription comes with 1 TB of OneDrive space (Family subscriptions with 1 TB per person).
I've been using it with restic + rclone successfully for years. It's not very fast, but works.
delusional
I'd recommend having a look at Hetzner's "storage box" products. It's hard to beat 4€ a month for 1TB of SSH accessible storage.
sunaookami
Seconding this, I use it to backup my server hosted elsewhere and it works flawlessly. They even support borg out of the box https://docs.hetzner.com/storage/storage-box/access/access-s...
copirate
SSH access on the storage boxes is quite limited:
> It is possible to get interactive SSH access, but this access is limited. It is not possible to have interactive access via port 22, but it is possible via port 23. There is no full shell. For example, it is not possible to use pipes or redirects. It is also not possible to execute uploaded scripts.
https://docs.hetzner.com/storage/storage-box/access/access-s...
https://docs.hetzner.com/storage/storage-box/access/access-s...
immibis
Accessible via SFTP - SSH's version of FTP.
Note they're intended for backup use and therefore don't have guaranteed uptime or throughput.
fourstepper
At what point would they effectively be selling you a VM with 1TB of storage for 4 EUR a month? :D
TiredOfLife
6x1TB for 10€ with 30 day cryptolocker protection comes close.
undefined
jjice
For personal use, at what point would one recommend using Borg over a regular rsync?
I currently use rsync to backup up a set of directories on a drive to another drive and a remote service (rsync.net). It's been working great, but I'm not sure if my use-case is just simple enough where this is a good solution, or if I'm missing a big benefit of Borg. I do envy Borg's encryption, but the complexity of a new tool tied with the paranoia of me maybe screwing up all my data has had me on edge a bit to make the leap. I don't have a ton of data to backup, say about 5TB at the moment.
zimpenfish
For me, the deduping and compression saves a lot of storage. My mail backup (17 backups covering the last 6 months) is originally 837GB, compressed to 312GB and dedupe'd to 19GB. Same with Postgres - 25GB to 7GB to 900MB.
You could probably use rsync's hard linking to save space on the mail backup but I'm not sure you'd get it as small without faffing about.
remram
Usual problem, if you delete/corrupt a file and find out two days later, your daily backup is not going to help you. Having more than one snapshot is very valuable.
Rsync is also very slow with lots of files, and doesn't deal with renamed files (will transfer again).
eikenberry
Rsync backups can be setup to deal with this. I have rsync setup with daily incremental backups, the main sync to a 'current' folder and the old version of changed files staying in a weekday named folder (eg. Monday). So I have a rotating 7 day period to recover files. On top of that I have a monthly long term backup of the last old version of that month. This provides an arbitraribly long monthly window to recover from. Rsync is very versatile.
remram
Yeah with enough scripting, you can rebuild a slow equivalent to a real backup program, that will also use 10x the disk space.
ibizaman
With rsync, you’re replicating only the last state. With borg, you can see all backups being made and rollback to any previous snapshot. This is true of a lot of backup solutions btw.
Concretely, if you inadvertently delete a file and this get rsynced, you cannot use the backup to restore that file. With borg you can.
mendym
is there a reason to use the borg encryption[1] over rclone crypt[2] or vise versa?
1. https://borgbackup.readthedocs.io/en/2.0.0b11/quickstart.htm... 2. https://rclone.org/crypt/
aborsy
Rclone crypt is not much related to Borg. That’s a tool for copying files from one machine to another, in this case encrypting before copying. That’s rsync, working with cloud.
Borg is a different tool, for backup. It deduplicates, encrypts, snapshots, checksums, compresses, … source directories into a single repository. It doesn’t work with files, rather blocks of data. It includes commands for repository management, like searching data, pruning or merging snapshots, etc. You will then transfer or sync the repository to wherever you want, with a tool such as rsync/SSH or rclone. Rclone is now natively supported, so that you don’t need to store the repository locally and on remote, rather back up directly to remote.
misanthr0pe
I would also wonder what the difference between this package and Restic is. as far as efficiency and encryption.
undefined
freeqaz
How good at deduping is this when encryption is enabled? I was looking at rsync.net and it killed me that they don't support encryption in a sane way.
djbusby
It's very sane: encrypt the bits, then send it to the host.
Curious what you think is not right with their methods.
freeqaz
Sure, but there is some requirement to not just blindly copy everything over-and-over, and that is where I've seen things get tricky before. If you enable encryption you have to re-upload the entire snapshot periodically.
It's annoying because if you have TBs of stuff that blows. I'm just curious what systems exist for incremental, encrypted backups that don't require full uploading new snapshots.
See here in the NOTE section. Re-reading this, it might a limitation of Duplicity. https://www.rsync.net/resources/howto/duplicity.html
elric
I've happily been writing borg backups to rsync.net for years. They have support for forcing borg in the ssh session using force-command, and borg has options that can prevent deletion (should the backup ssh key be compromised).
Overall it's a robust solution that isn't too painful to setup.
unbrice
I second this. I was looking for a solution that prevented a compromised host from deleting its own backups. Forcing the command as you mentioned works for rsync.net, and its snapshots also provide a protection against fat finger errors.
aquafox
Also BorgBackup user here: I'm running it on a Raspberry Pi to backup important documents to a Hetzner storage box via ssh. The Pi als runs OpenMediaVault to provide a SMB share on my home LAN. So whenever I scan a new document, just put it on the SMB share and from there it's backed up automatically every day.
wzyboy
I've always been doing "two-pass" backups to achieve "3-2-1" goal: first pass is to run BorgBackup to backup devices to my home server. The second pass is to use rclone to transfer the repos on home server to an object storage service (B2).
With rclone support built-in, the setup would be much easier.
cl3misch
I think this is heavily discouraged? Instead you should have multiple separate borg repos to minimize risk of misconfiguration and data corruption.
cl3misch
I managed to find the source for my statement:
https://borgbackup.readthedocs.io/en/stable/faq.html#can-i-c...
It is not "heavily" discouraged. But you have to pay extra attention to perform a clean copy of the borg repo's files, and ideally check both instances regularly for integrity. I would assume it's easy to forget validating the "cold storage" copy of your borg repo in practice.
Get the top HN stories in your inbox every day.
BorgBackup user here and really happy. It was a set and forget for me and after 7 years, the deduplicated backup is still working flawlessly each week. I recommend pairing it with borgmatic [1], which helps to design away some of the complexities of the underlying borg backup.
[1]: https://github.com/borgmatic-collective/borgmatic