Author Topic: Keeping your stuff in Sync ...  (Read 2484 times)

0 Members and 1 Guest are viewing this topic.

Offline Mad Penguin

  • Administrator
  • Hero Member
  • *****
  • Posts: 1436
  • Karma: 10023
  • Gender: Male
    • View Profile
    • Awards
Keeping your stuff in Sync ...
« on: April 03, 2010, 01:09:56 am »
One of the problems I see more and more of these days are faulty hard drives, boy is it painful when they go belly up. "Ah" I hear you say, you should be using RAID .. well .. yes and no. Firstly if your box gets stolen or bursts into flames - it's all gone, and secondly, RAID is no guarantee. And thirdly, all it takes is rm -rfv ~/junk /* , one errant space and you're all gone!

Six months ago I put in a new workstation with a RAID 5 configuration, 3 live disks and a hot swap. Two months ago I had to recover everything from backups when one of the disks failed completely (complete electronic failure) and a second (at the same time) came up with so many bad blocks the RAID just couldn't recover. Ho Hum.

Anyway, if you're developing 'stuff' you want regular backups and although CVS sounds good in principle, it's a pain to use not to mention setting it up and running it may be more trouble than you want.

Solution; enter "rsync"

Everyone knows (?) rsync is a nice backup tool, but one of the lesser know features is it's ability to do incremental (cvs style) backups. Ideally you run this against a remote server so in any eventuality your work is all intact, indeed if you run this from cron every minute you'll never lose anything worth talking about .. :)

And, another feature maybe not so obvious, although rsync 'can' use it's own server, it's just as happy running against a SSH server, so if you already have remote access to a SSH server - you're already set up!

All you need is a script, something like this;

Code: [Select]
#!/bin/bash
BACK=`date "+%A_%d_%B_%Y_%H_%M"`
OPTS="--delete-excluded --delete --backup --backup-dir="../${BACK}" -ravzltO"
HOST="<remote server name or ip>"
USER="<login id for remote server>"
ARCH="<archive folder location on remote server>"
rsync ~ ${OPTS} ${USER}@${HOST}:/${ARCH}/current

What does it do;

Every time it runs it should update an image of everything in/under your home folder on the remote server. Any files that are updated are saved before the update, and put into a folder which takes it's name from today's date/time. So in the archive folder on the remote server you will have a folder called "current" which is essentially a copy of your home folder, and a series of folders with dates for names which are all your edits / old copies / changed files. So at any time you can recover your home folder from "current", or you can search for previous / old versions / deleted files in the various dated folders. Note that this effectively works like a recoverable trash can as it will also keep copies of deleted files (!)

Making it automatic;

Once you have this running, you will need to make sure that authentication with the remote server is via certificate and not password. To do this you need to copy your public key (typically ~/.ssh/id_rsa.pub) to the remote server and install it in; ~/.ssh and call it; authorized_keys2

To verify this works, you should be able now to ssh to the remote server, and it should log you in without you having to enter a password. This is critical if you want your backup to work unattended.

Now add your script to cron (as 'you' , *not* as 'root');

Code: [Select]
crontab -e

Enter a line;

Code: [Select]
* * * *  *  <full path to your script>

Cron can be an interesting tool to debug .. to check it, make sure your machine has a working email setup;

Code: [Select]
echo "Hello World" | mail <your email address>

Assuming this works add, run crontab -e again and insert this as the first line;

Quote
MAILTO=<your mail address>

And it will mail you the results of the rsync every time it fires. Once you're happy with things and that it's doing what you want, add the following to the rsync line in your crontab;

Quote
> /dev/null

This will redirect standard output off to a black hole, so you won't get all the emails .. however it will not redirect stderr (any errors), so if the rsync fails  it will email you the appropriate error message.

Any questions feel free .. you can never make too many backups!

 


SimplePortal 2.3.3 © 2008-2010, SimplePortal