Script: Run rsnapshot backups only once and rollback failed backups using rsnapshot-once

June 28, 2013 · Linux, Scripting, Synchronization

Contents

I use rsnapshot to backup all of my data to my HTPC and home server (the home partition, office documents and the root file system). While rsnapshot is not as shiny as other backup tools, it is very flexible and effective: rsnapshot is based on rsync and makes hardlink-based backups (like cp -al), i.e. backups that point to the same inode on the disk if a file in consecutive backups is identical (much like SIS in deduplication).

However, rsnapshot is meant to be triggered by cronjobs and is built for always-on server machines rather than for lid-open-lid-close-type machines like laptops: That means that rsnapshot must be scheduled to run at a certain time (no retries!) and is not prone sudden system shutdowns. Furthermore, it does not detect failures and simply leaves unfinished backups as if they were complete. That in turn leads to more disk space being used for the backups, because the last complete backup is not really complete.

I wrote a little helper script to fix exactly this behavior: rsnapshot-once makes sure that (1) rsnapshot is only called if a backup is necessary (once every 24h for ‘daily’, once ever 7 days for ‘weekly’, …) even if rsnapshot-once is called multiple times, and (2) that crashed/interrupted backup runs are rolled-back before starting a new backup run.

1. What is rsnapshot-once?

rsnapshot-once is a wrapper for rsnapshot to ensure that daily, weekly and monthly tasks are run only once in the respective time period, i.e. it ensures that ‘weekly’ backups are executed only once a week, regardless how often rsnapshot-once is called.

rsnapshot-once accepts the same parameters as rsnapshot and uses the same config file. No additional configuration is needed.

So instead of running something like this:

Run rsnapshot

rsnapshot -c /etc/rsnapshot.home.conf daily

You would rather run this:

Run rsnapshot-once instead

rsnapshot-once -c /etc/rsnapshot.home.conf daily

Similarly, simply replace rsnapshot by rsnapshot-once in the crontab, if you run backups regularly as a cronjob. Instead of this:

Crontab entry for rsnapshot

# Hope that your computer is on at 10am
0 10 * * * rsnapshot -c /etc/rsnapshot.home.conf daily

Write this:

Crontab entry for rsnapshot-once

# daily (try every full hour!)
0 * * * * rsnapshot-once -c /etc/rsnapshot.home.conf daily

2. What’s the difference?

There are a couple of things that rsnapshot-once adds to the regular rsnapshot-behavior:

Run multiple times: For the intervals ‘daily’, ‘weekly’ and ‘monthly’, rsnapshot-once will first check the timestamp of the last successful backup (daily.0, weekly.0 or monthly.0) before running rsnapshot. So rsnapshot-once will only call rsnapshot if the timestamp of the ‘daily.0’ directory is more than 24 hours old.
Rollbacks of failed backups: Before running rsnapshot, rsnapshot-once checks whether the last backup was complete. If it wasn’t, the last directory (e.g. daily.0) is deleted and consecutive directories are renamed (daily.1 -> daily.0, …). rsnapshot-once uses its own pidfile to check if a previous backup run has been interrupted.
15 minutes after boot/standby: To make sure that a backup is not starting right after the boot process, rsnapshot-once checks the uptime and the resume time before running rsnapshot. Only if both are greater than 15 minutes, rsnapshot is called.

3. Installation

Simply download rsnapshot-once, place it in your local bin directory (e.g. /usr/local/bin) and make it executable. If you haven’t already, install the PHP command line interpreter. Once this is done, it can be called from the command line or in cronjobs:

sudo apt-get install php5-cli
wget -O /usr/local/bin/rsnapshot-once https://heckel.io/uploads/2013/06/rsnapshot-once
chmod +x /usr/local/bin/rsnapshot-once

4. Example Usage

I use rsnapshot and rsnapshot-once for backing up the home-partition (/home), the root partition (/) and my office documents (/home/user/Office). All of them are backed up weekly and monthly, but the office documents are additionally backed up daily (because they change more often).

I set up my cronjobs to run rsnapshot-once like this:

# DAILY
# Try a couple of times a night, 8-11pm
45 21,22,23 * * * rsnapshot-once -c /etc/rsnapshot.office.conf daily

# WEEKLY
# Try for 3 consecutive days, a couple of times a night, Mon-Wed 8-11pm
0 20,21,22,23 * * 1,2,3 rsnapshot-once -c /etc/rsnapshot.office.conf weekly
15 20,21,22,23 * * 1,2,3 rsnapshot-once -c /etc/rsnapshot.root.conf weekly
30 21,21,22,23 * * 1,2,3 rsnapshot-once -c /etc/rsnapshot.home.conf weekly

# MONTHLY
# Try on 4 consecutive days, a couple of times a night, 1st-4th of each month
59 22,23 1,2,3,4 * * rsnapshot-once -c /etc/rsnapshot.office.conf monthly
58 21,22 1,2,3,4 * * rsnapshot-once -c /etc/rsnapshot.root.conf monthly
57 20,21 1,2,3,4 * * rsnapshot-once -c /etc/rsnapshot.home.conf monthly

First/any run: When rsnapshot-once is run, it writes to the same log file as rsnapshot (if defined in the config file). The output of a successful run for my daily office documents looks like this:

[23:42:57] ## STARTING BACKUP ######################
[23:42:57] $ rsnapshot-once '-c' '/etc/rsnapshot.office.conf' 'daily'
[23:42:57] Config read from: /etc/rsnapshot.office.conf
[23:42:57] - logfile = /var/log/rsnapshot.office.log
[23:42:57] - snapshot_root = /backups/office/
[23:42:57] Checking rsnapshot-once pidfile at /backups/office/.rsnapshot-once.pid ... Does not exist. Last backup was clean.
[23:42:57] Checking delays (minimum 15 minutes since startup/wakeup) ...
[23:42:57] - Computer uptime is 387.5 minutes. THAT'S OKAY.
[23:42:57] - Computer resume time is 48.0 minutes. THAT'S OKAY.
[23:42:57] Newest backup for 'daily' at /backups/office/daily.0 was at 27/Jun/2013, 23:42:53.
[23:42:57] Last run is 24.0 hour(s) ago (min. is 23 hours).
[23:42:57] Writing rsnapshot-once pidfile (PID 14469) to /backups/office/.rsnapshot-once.pid.
[23:42:57] NOW RUNNING JOB: rsnapshot '-c' '/etc/rsnapshot.office.conf' 'daily' 2>&1
[23:42:58] /usr/bin/rsnapshot -c /etc/rsnapshot.office.conf daily: started
[23:42:58] echo 14486 > /var/run/rsnapshot.pid
[23:42:58] /bin/rm -rf /backups/office/daily.6/
[23:43:10] mv /backups/office/daily.5/ /backups/office/daily.6/
[23:43:10] mv /backups/office/daily.4/ /backups/office/daily.5/
[23:43:10] mv /backups/office/daily.3/ /backups/office/daily.4/
[23:43:10] mv /backups/office/daily.2/ /backups/office/daily.3/
[23:43:10] mv /backups/office/daily.1/ /backups/office/daily.2/
[23:43:10] /bin/cp -al /backups/office/daily.0 /backups/office/daily.1
[23:43:21] /usr/bin/rsync -ax --delete --numeric-ids --relative --delete-excluded /home/user/Office /backups/office/daily.0/office/
[23:43:44] touch /backups/office/daily.0/
[23:43:44] rm -f /var/run/rsnapshot.pid
[23:43:44] /usr/bin/rsnapshot -c /etc/rsnapshot.office.conf daily: completed successfully
[23:43:44] Removing rsnapshot-once pidfile at /backups/office/.rsnapshot-once.pid (CLEAN EXIT).
[23:43:44] Rotating log ...
[23:43:44] ## BACKUP COMPLETE ######################

Duplicate run: If the same command is again right after this backup, rsnapshot will not be called, because the las successful backup was less than 24h ago:

[23:47:02] ## STARTING BACKUP ######################
[23:47:02] $ rsnapshot-once '-c' '/etc/rsnapshot.office.conf' 'daily'
[23:47:02] Config read from: /etc/rsnapshot.office.conf
[23:47:02] - logfile = /var/log/rsnapshot.office.log
[23:47:02] - snapshot_root = /backups/office/
[23:47:02] Checking rsnapshot-once pidfile at /backups/office/.rsnapshot-once.pid ... Does not exist. Last backup was clean.
[23:47:02] Checking delays (minimum 15 minutes since startup/wakeup) ...
[23:47:02] - Computer uptime is 391.6 minutes. THAT'S OKAY.
[23:47:02] - Computer resume time is 52.1 minutes. THAT'S OKAY.
[23:47:02] Newest backup for 'daily' at /backups/office/daily.0 was at 28/Jun/2013, 23:43:44.
[23:47:02] Job does NOT need to run. Last run is only 0.1 hour(s) ago (min. is 23 hours). EXITING.
[23:47:02] ## BACKUP ABORTED #######################

Rollback run after interrupted backup: If the command is run after a failed or interrupted backup (e.g. due to a system shutdown), the output looks something like this:

[00:01:53] ## STARTING BACKUP ######################
[00:01:53] $ rsnapshot-once '-c' '/etc/rsnapshot.office.conf' 'daily'
[00:01:53] Config read from: /etc/rsnapshot.office.conf
[00:01:53] - logfile = /var/log/rsnapshot.office.log
[00:01:53] - snapshot_root = /backups/office/
[00:01:53] Checking rsnapshot-once pidfile at /backups/office/.rsnapshot-once.pid ... Exists. PID 14678 not running. Script crashed before.
[00:01:53] Cleaning up unfinished backup ...
[00:01:53] - Deleting /backups/office/daily.0 ... DONE
[00:02:00] - Moving /backups/office/daily.1 to /backups/office/daily.0 ... DONE
[00:02:00] - Moving /backups/office/daily.2 to /backups/office/daily.1 ... DONE
[00:02:00] - Moving /backups/office/daily.3 to /backups/office/daily.2 ... DONE
[00:02:00] - Moving /backups/office/daily.4 to /backups/office/daily.3 ... DONE
[00:02:00] - Moving /backups/office/daily.5 to /backups/office/daily.4 ... DONE
[00:02:00] - Moving /backups/office/daily.6 to /backups/office/daily.5 ... DONE
[00:02:00] Checking delays (minimum 15 minutes since startup/wakeup) ...
[00:02:00] - Computer uptime is 406.6 minutes. THAT'S OKAY.
[00:02:00] - Computer resume time is 67.0 minutes. THAT'S OKAY.
[00:02:00] Newest backup for 'daily' at /backups/office/daily.0 was at 27/Jun/2013, 23:50:53.
[00:02:00] Last run is 24.2 hour(s) ago (min. is 23 hours).
[00:02:00] Writing rsnapshot-once pidfile (PID 14782) to /backups/office/.rsnapshot-once.pid.
[00:02:00] NOW RUNNING JOB: rsnapshot '-c' '/etc/rsnapshot.office.conf' 'daily' 2>&1
[00:02:10] Removing rsnapshot-once pidfile at /backups/office/.rsnapshot-once.pid (CLEAN EXIT).
[00:02:10] Rotating log ...
[00:02:11] ## BACKUP COMPLETE ######################