Incremental backups with rsync

There are some good resources guiding people to make shell scripts for creating incremental backups using rsync, but it’s still a lot to wade through if you’re not very familiar with some of the ins and outs of *nix terminal commands.

** NOTE: I’m still testing these scripts to make sure they all work. In theory, and by appearance they do. Use at your own risk (you’ve been warned =) **

These two scripts will give you daily backups for a week, and three weekly backups (from the incremental).

1) make backups on a separate partition, on a separate disk, on a separate server. Don’t have the money for buying a new server? Well, your money can buy you more than you realize. You’ll do just fine to set-up a PIII ~500MHz with a reasonable amount of ram (512MB) and a new hard disk drive. 200GB drives are going for less than $100 these days, so you’ve got yourself a backup server!

Get on with it! Two scripts – one for the daily tasks, one for the weekly. Ok – four. One will set-up your cron jobs, but it’s so easy it won’t really count. I’m using BASH for the examples, but T/CSH or any other scripting language should be fine with some modifications

daily.sh

#!/bin/sh

##################################################
#needs to be run as root (to preserve permissions)
# NOTE: unlike the weekly script that saves backups
# on the same day every week (e.g., every Sunday),
# this script makes backups that are X days old
# instead of indicating the day of the week they
# were made (daily.0 is made every day, whereas
# weekly.1 is made every Sunday
##################################################

#the mount point of the drive using for your backups
BACKUP_DIR='/backup'

#mount drive read-write so we can back-up to it.
#you'll want to change the entry in your fstab so it's mounted as read-only by default
/bin/mount -o remount,rw $BACKUP_DIR/

## rotate-out the daily backups
/bin/rm -rf $BACKUP_DIR/daily.6
/bin/mv $BACKUP_DIR/daily.5 $BACKUP_DIR/daily.6
/bin/mv $BACKUP_DIR/daily.4 $BACKUP_DIR/daily.5
/bin/mv $BACKUP_DIR/daily.3 $BACKUP_DIR/daily.4
/bin/mv $BACKUP_DIR/daily.2 $BACKUP_DIR/daily.3
/bin/mv $BACKUP_DIR/daily.1 $BACKUP_DIR/daily.2
/bin/mv $BACKUP_DIR/daily.0 $BACKUP_DIR/daily.1

#start with a fresh copy...
/bin/mkdir $BACKUP_DIR/daily.0

#the good stuff. Note that you need to set-up a public key and allow list between
#the two computers (will cover in a later post)
/usr/bin/rsync -e ssh -vcpogrt --numeric-ids --delete --exclude-from=/path/to/backup_exclude_list.txt --link-dest=$BACKUP_DIR/daily.1 root@serverIPaddress:/ $BACKUP_DIR/daily.0 > /tmp/rsyncAll

/bin/echo "n" >> /tmp/rsyncAll

#send you an email with any messages (errors) returned by the script.
mail -s "rsyncAll backupserver from mainserver" -c youremail@domain.ext root


exclude list
You’ll need this for sure on your system. I haven’t quite figured out all the things that need to be excluded, but I’ve put most of the important ones here. Note that I’m using Fedora (FC4), so your milage may vary. Make sure to put each entry on its own line.

/proc/
/tmp/
/mnt/
/backup/
/media/
/media/backup/
/sys/devices/pci*


weekly.sh
The weekly script is real easy. just some copying/moving.


#!/bin/sh
####################################################
#needs to be run as root (to preserve permissions)
# Run this script BEFORE the daily script ...
# Weekly backups are taken on the same day every week
# and are NOT weekly from the current day. Note that
# distinction.
####################################################
BACKUP_DIR='/backup'

#mount drive so we can back-up to it.
/bin/mount -o remount,rw $BACKUP_DIR/

#previous third week's stuff is outta here!
/bin/rm -rf $BACKUP_DIR/weekly.2

/bin/mv $BACKUP_DIR/weekly.1 $BACKUP_DIR/weekly.2
/bin/mv $BACKUP_DIR/weekly.0 $BACKUP_DIR/weekly.1

# 6-day-old data now the one-week-old data (weekly.0)
/bin/cp -al $BACKUP_DIR/daily.6 $BACKUP_DIR/weekly.0

#daily.6 is now vacant and will be filled by daily script

#mount drive read-only so nobody can touch it
/bin/mount -o remount,ro $BACKUP_DIR/


Finally, you’ll probably want to add some cron jobs to your list…
By default, there were no jobs in my root’s crontab, so it’s no problem to enter the following into a file:

EDIT: simple mistake on the weekly.sh entry: should be 0 0 * * 0 to run every Sunday at midnight.
0 0 * * 0 /bin/sh /path/to/scripts/weekly.sh
30 0 * * * /bin/sh /path/to/scripts/daily.sh

All this means is the following: Every Sunday at midnight, run the weekly script (you’ll want to do this BEFORE your daily script runs so that the daily.6 gets moved over to the weekly set before it gets overwritten by the job that’s to run thirty minutes later. The numbers and stars are basically used to tell cron when to run the scripts. Just run a Google search to find a LOT more detailed information.

Enter it by typing into the terminal (as root): crontab /path/to/crontabfile.txt
you can verify your scheduled jobs by running: crontab -l