I’m curious what people are using to monitor their backups? I have Borg running on a cronjob, but checking logs periodically is getting tedious, so I’d like to automate that if possible
I recommend a dead man’s switch like Healthchecks.io, which can be selfhosted for free. Whenever you have something that’s regularly occurring, add an extra callout to your unique Healthchecks callout UUID as part of the automation, and Healthchecks will send you a notification if something misses its callout schedule. You can also attach whatever data (e.g. a log) to the callout so you can look back through the run history. IIRC Borg will give you a non-zero return code if it detects problems, so you can send e.g.
https://hc-ping.com/your-uuid-here/$?and a non-zero code will signal a notification as well (more examples here).Also, Borgmatic is really easy to use for managing Borg repos. There’s a lot of configuration options (including Healthchecks.io integration) but you can delete like 90% of it for normal usecases.
I use healthchecks for all my scripts and making sure my docker containers and services are up. If anything goes down or errors then I get sent a telegram message immediately.
If you want to do this, what you probably want is to pump your logs into a log drain, something like betterstack is good. They then allow you to set up discrepancy thresholds and can send you emails when something seems to be out of the ordinary. There’s probably a self hosted thing that works the same way but I’ve never found a simple setup. You can do the whole Prometheus, influxdb, grafana setup but imo it’s too much work, and then you still have to set up email smtp separate from that.
Use unit files and add on failure a notification per mail or whatever.
You still should test the backups periodically.
I just use ’’’borg list [mypath] | tail 1’’’ in my zshrc on my desktop to see the last backup date. I guess you could make it work on server if you mail that output to yourself or something similar. Maybe add it to your cronjob?
something that important I’d put into the login banner as well.
+1 for healthchecks.io, and I also use ntfy.sh so every morning I can wake up to a warm, fuzzy, “backup successful” notification.
Are you looking for something for a headless server or something for a system with a GUI (eg a desktop)?
Headless. It’s all running fine in the background. For now…
I use Borgbase as one of my backup destinations and you can set it to send an email if they haven’t received a new backup for a number of days.
