Cameron Perry | Archives

There are some good resources guiding people to make shell scripts for creating incremental backups using rsync, but it’s still a lot to wade through if you’re not very familiar with some of the ins and outs of *nix terminal commands.

** NOTE: I’m still testing these scripts to make sure they all work. In theory, and by appearance they do. Use at your own risk (you’ve been warned =) **

These two scripts will give you daily backups for a week, and three weekly backups (from the incremental).

1) make backups on a separate partition, on a separate disk, on a separate server. Don’t have the money for buying a new server? Well, your money can buy you more than you realize. You’ll do just fine to set-up a PIII ~500MHz with a reasonable amount of ram (512MB) and a new hard disk drive. 200GB drives are going for less than $100 these days, so you’ve got yourself a backup server!

Get on with it! Two scripts – one for the daily tasks, one for the weekly. Ok – four. One will set-up your cron jobs, but it’s so easy it won’t really count. I’m using BASH for the examples, but T/CSH or any other scripting language should be fine with some modifications

daily.sh

#!/bin/sh

##################################################
#needs to be run as root (to preserve permissions)
# NOTE: unlike the weekly script that saves backups
# on the same day every week (e.g., every Sunday),
# this script makes backups that are X days old
# instead of indicating the day of the week they
# were made (daily.0 is made every day, whereas
# weekly.1 is made every Sunday
##################################################

#the mount point of the drive using for your backups
BACKUP_DIR='/backup'

#mount drive read-write so we can back-up to it.
#you'll want to change the entry in your fstab so it's mounted as read-only by default
/bin/mount -o remount,rw $BACKUP_DIR/

## rotate-out the daily backups
/bin/rm -rf $BACKUP_DIR/daily.6
/bin/mv $BACKUP_DIR/daily.5 $BACKUP_DIR/daily.6
/bin/mv $BACKUP_DIR/daily.4 $BACKUP_DIR/daily.5
/bin/mv $BACKUP_DIR/daily.3 $BACKUP_DIR/daily.4
/bin/mv $BACKUP_DIR/daily.2 $BACKUP_DIR/daily.3
/bin/mv $BACKUP_DIR/daily.1 $BACKUP_DIR/daily.2
/bin/mv $BACKUP_DIR/daily.0 $BACKUP_DIR/daily.1

#start with a fresh copy...
/bin/mkdir $BACKUP_DIR/daily.0

#the good stuff. Note that you need to set-up a public key and allow list between
#the two computers (will cover in a later post)
/usr/bin/rsync -e ssh -vcpogrt --numeric-ids --delete --exclude-from=/path/to/backup_exclude_list.txt --link-dest=$BACKUP_DIR/daily.1 root@serverIPaddress:/ $BACKUP_DIR/daily.0 > /tmp/rsyncAll

/bin/echo "n" >> /tmp/rsyncAll

#send you an email with any messages (errors) returned by the script.
mail -s "rsyncAll backupserver from mainserver" -c youremail@domain.ext root


exclude list
You’ll need this for sure on your system. I haven’t quite figured out all the things that need to be excluded, but I’ve put most of the important ones here. Note that I’m using Fedora (FC4), so your milage may vary. Make sure to put each entry on its own line.

/proc/
/tmp/
/mnt/
/backup/
/media/
/media/backup/
/sys/devices/pci*


weekly.sh
The weekly script is real easy. just some copying/moving.


#!/bin/sh
####################################################
#needs to be run as root (to preserve permissions)
# Run this script BEFORE the daily script ...
# Weekly backups are taken on the same day every week
# and are NOT weekly from the current day. Note that
# distinction.
####################################################
BACKUP_DIR='/backup'

#mount drive so we can back-up to it.
/bin/mount -o remount,rw $BACKUP_DIR/

#previous third week's stuff is outta here!
/bin/rm -rf $BACKUP_DIR/weekly.2

/bin/mv $BACKUP_DIR/weekly.1 $BACKUP_DIR/weekly.2
/bin/mv $BACKUP_DIR/weekly.0 $BACKUP_DIR/weekly.1

# 6-day-old data now the one-week-old data (weekly.0)
/bin/cp -al $BACKUP_DIR/daily.6 $BACKUP_DIR/weekly.0

#daily.6 is now vacant and will be filled by daily script

#mount drive read-only so nobody can touch it
/bin/mount -o remount,ro $BACKUP_DIR/


Finally, you’ll probably want to add some cron jobs to your list…
By default, there were no jobs in my root’s crontab, so it’s no problem to enter the following into a file:

EDIT: simple mistake on the weekly.sh entry: should be 0 0 * * 0 to run every Sunday at midnight.
0 0 * * 0 /bin/sh /path/to/scripts/weekly.sh
30 0 * * * /bin/sh /path/to/scripts/daily.sh

All this means is the following: Every Sunday at midnight, run the weekly script (you’ll want to do this BEFORE your daily script runs so that the daily.6 gets moved over to the weekly set before it gets overwritten by the job that’s to run thirty minutes later. The numbers and stars are basically used to tell cron when to run the scripts. Just run a Google search to find a LOT more detailed information.

Enter it by typing into the terminal (as root): crontab /path/to/crontabfile.txt
you can verify your scheduled jobs by running: crontab -l

Earlier this week I had the unpleasant task of trying to fix/update the NVidia drivers on my Fedora workstation. Video was working fine. What was giving me problems was the hardware 3D support, which is needed by the uber expensive visualization app I use for data analysis.

I’ve never had a pleasant, much less easy, time upgrading the video drivers on this machine. It started with the RedHat Enterprise Linux distribution – better known as “Always at least two full versions behind current,” or, “This software base is at least two years behind other distributions.” Dell suggests you use their “custom” NVidia drivers. NVidia says to use theirs. Neither worked very well, though Dell’s seemed to be more problematic than it was worth.

Fast forward to Fedora.
This Link, though it’s probably the best option, didn’t help – the instructions didn’t work because Yum couldn’t resolve some pretty idiotic dependency issues with my kernel. Next option: the actual NVidia installer. To be honest, it didn’t work the first time around, which is why I tried the Yum installer, but I thought I’d give it another shot.

• Download the Linux NVidia driver install script.
• Change your /etc/inittab to run level 3 (from 5 in the first line of executable code), reboot (or just log-out, hit control-alt-F1 and as root run telinit 3).
• As root run the script. At this point, if I recall correctly, you’d actually be able to telinit 5 and use full 3D capabilities (maybe modprobe nvidia would be required first). I ran into a problem that the NVidia ftp site was down/not responding, so it couldn’t download a pre-compiled profile for the video drivers. Just answer “Yes” when it asks if you want it to compile it for you.

The problem: if you reboot (run level 5), no more nvidia driver. You’ll see an error that the .ko file can’t be found at boot. So something is happening between the install and the boot process. glxgears won’t work – nothing – which is as expected.

• To make a long story short – look in your kernel’s drivers directory. The script doesn’t put the .ko files in the correct places – at least not initially. What you need to do is make sure that the nvidia.ko file can be found, in one way or another in the …/kernel/drivers/ and the …/kernel/drivers/video/ directories… yes, that’s both of them.
• open, as root, the /etc/inittab and change the run level back to 5. save. close.

reboot.

Here’s another link that ultimately wasn’t too useful to me, but gave me some ideas:Here.

Hopefully somebody will find these instructions even slightly useful