Peter's Blog - Nodes for backup

Disk Full

I set up a cron job to backup my web sites to my Mythical Convergence Box, just a simple rsync of the subversion repository and, to be safe, /var/www. The latter was 500Mbytes.

Next morning the cron job had run and my server was inaccessable. Oops, with 10g of WoW and 10g of family photos I had filled the main disk up, my 250g Myth disk being mounted on /var/lib/mythtv. The box hadn't crashed but it was in a kind of coma, you ask it to do something and it went urgh.

I had to reboot into recovery mode and move the photos to the mythtv disk, leaving me 68%. Observations:

Interesting how hard the system failed when the disk filled up. Have to be careful.
Linux recovery mode is actually useful! Whenever I've tried the (admittedly pre-vista) Windows recovery modes it has spent a quarter of an hour loading drivers for obscure disk controllers, then I've tried the automated recovery that never works, then I've rebooted, I've waited another quarter hour, then it's given me something with the raw power of a 1985 dos prompt, then I've given up and reformatted the disk.

Last night my cron job worked fine and my websites are so thoroughly backed up that my head is spinning.

Out with Rsync, in with Unison

I had my rsync scripts set up and running. I had backup and get scripts so I could backup my work to a server, go home, download the changes, do some work, upload the changes, download them again at work. Who needs a flash drive?

However there's this thing called 'finger trouble' that happens sometimes: you forget to backup the changes on one machine or another and suddenly, bam, your work vanishes. At least that is what I think happened, all I know is that twice now I have lost some work. Running backup and get scripts to synchronise two pc's against a server is too error prone for me.

So I set up unison which, once running, should be a less error prone solution. It will synchronise the files on my home and work pc's with the files on the server. Changes to the files may be propogated either way, depending on the time stamps. I can change the files on either pc and unison will help me resolve any conflicts I create. While it is synchronising it will prompt me with each file that has changed and ask me what to do with it so I can see if I am about to overwrite an evening's work.

I have the following unison profile file set up in my .unison directories on both pc's:

#
# Unison profile for project X
#
root = /cygdrive/c/projects/X
root = ssh://me@myserver.com//home/pcw/X

ignore = Name Downloads/*
ignore = Name Backup/*
ignore = Name Backups/*
ignore = Name Tmp/*
ignore = Name *.map
ignore = Name *~
ignore = Name *.obj
ignore = Name *.o
ignore = Name *.i

backup = Name *
backupdir = f:\users\pcw\BACKUP\X

If the profile file is called X.prf then I can synchronise this project by running

unison X

Because the profile is in my private .unison directory I don't even need to cd to the project directory.

The 'backup' and 'backupdir' lines at the end of the profile tell unison to maintain an extra local backup, in this case on our main file server such that the backups fall under our company tape backup procedure.

It has worked for a day now with fewer problems than rsync gave (and ironically unison uses rsync in it's underpinnings) and it is fast, taking less than a minute to work on 491 files.

To maintain straight backups unison would be overkill but where changes could be made to the files in more than one place it is looking good.

Of course, I could use unison with a flash drive rather than a file server but I don't have to remember to carry the server around with me.

And I know about synctoy but unison works easily through an ssh tunnel.

Backing up from Windows to a linux box

Another backing up setup is under construction. This time I'm backing up my project developed on a windows box to a linux server using rsync. It is using ruby as a scripting language. If this script is invoked thus:

ruby Backup.rb

then this script will merely update a simple backup of the source directory on the server. rsync is very fast and will only spend time uploading files that have actually changed and also it will only upload the changes to those files. I am using this on a 30M source archive and it takes less than a minute to update the backup. The backup files are just copies of the original files so easy to browse, diff, restore.

If the script is invoked thus:

ruby Backup.rb --backup

then it will use rsync's magic link-dest option to create historical archive of backups. What the script will do is this:

Move the current main backup to an archive directory and give that archive directory a name that is date/time stamped.
Create a new backup. Where files have not changed from the previous backup, the new backup will contain a link to the existing copy of the file in the archive, rather than consume more file space. The new backup directory will only contain files that are new or have been changed.

There is nothing in the script to limit the number of backups but they can be manually pruned every few months as required.

This is the script:

   1  #
   2  # This script is used to backup the source code to an offsite server
   3  #
   4  # If the parameter '--backup' is provided then create a backup directory
   5  #
   6  strMeAtMine = "me@myserver.org"
   7  strLocalFolder = "/cygdrive/c/projects/757/"
   8  strTargetFolder = "/home/pcw/757"
   9  strBackupFolder = "/home/pcw/Backup/757"
  10  
  11  #
  12  # If asked to backup then move previous backup to a backup directory
  13  # given a name related to date/time.
  14  #
  15  if ARGV.index( '--backup')
  16    bBackup = true
  17  
  18    strTimeStamp = Time.now.strftime( '%Y-%m-%d-%H-%M')
  19  
  20    strBackupDir = "#{strBackupFolder}/#{strTimeStamp}"
  21  
  22    strCommand = "ssh #{strMeAtMine} mv #{strTargetFolder} #{strBackupDir}"
  23  
  24    system( strCommand)
  25  else
  26    bBackup = false
  27  end
  28  
  29  #
  30  # Determine location of rsync exluded file list
  31  #
  32  strDir = File.dirname( __FILE__)
  33  strExcludeFile = File.join( strDir, "rsync-exclude.txt")
  34  
  35  #
  36  # Build command to invoke in an array of strings since this allows me to comment what
  37  # each parameter does.
  38  #
  39  strCommand = [ "rsync",
  40  #  	"-n",                                 # -n = dry run
  41    	"-v",                                 # verbose
  42                  "-a",                                 # archiving options
  43                  "--delete",                           # delete files no longer used from target
  44                  "--chmod=u=rwX",                      # set target file permissions
  45                  "--exclude-from=#{strExcludeFile}",   # exclude rubbish
  46                  "--delete-excluded",                  # delete excluded files from target
  47                  "-e ssh",                             # use ssh tunnel
  48                  strLocalFolder,                       # source
  49                  "#{strMeAtMine}:\"#{strTargetFolder}\"" # target
  50              ]
  51  
  52  if bBackup
  53    #
  54    # Backing up so instead of uploading everything again, link to files in the
  55    # backup directory where there are no changes.
  56    #
  57    strCommand.insert( -3, "--link-dest=#{strBackupDir}")
  58  end
  59  
  60  system( strCommand.join(" "))

Since this is a backup of files from a windows box to linux I used the "--chmod=u=rwX" to specify simple file permissions on the linux end. Without this rsync was tending to create files with no access permissions for anyone.

The 'rsync-exclude.txt' file lives in the same directory as the script and the script uses some magic to find it. This is a list of stiff that doesn't need backing up:

/Downloads
*.map
Backup/*
Backups/*
*~

This uses the cygwin version of rsync which uses ssh to talk to the remote server. I have ssh set up with key files so I don't need to enter passwords.

Vista backup

Vista has a new and improved backup facility. This is what Microsoft has to say:

"The file backup feature in Windows Vista is targeted at the core consumer audience. To savvy users, it will appear quite simple, probably too simple for you to use. For most consumers, however, it will hopefully be the right balance of complexity and functionality to get them backing up their files, which is something virtually none of them did in Windows XP, no matter how capable the tool was. For core consumers, we wanted to eliminate the confusion caused by selecting individual folders and files from a long tree of checkboxes, trying to find or guess where other users kept their files, predicting where files would be stored in the future, etc. These choices added too much complexity to the backup process and discouraged users from using NTBackup."

I tried to use it, clicked ok in the UAP box and got this:

I'm not sure where this puts me in the demographics. I'm savvy enough to see that it doesn't look that simple to get it running and as a normal consumer I'd say the available functionality won't get me backing up my files.

Mozy

Following my ruminations on online backup I'm giving mozy a try with the 2G free space plan. I selected 800M of stuff to backup and it's still working on uploading it. The signup wasn't problem free with one or two timeouts and the setup program was disdainful about my upload speed, saying it wasn't good enough but allowing me to proceed anyway. It will take a few days to backup everything. The backup seems to be interrupted whenever I use the computer (like Google Desktop Search) probably because Windows is so poor at multitasking disk intensive applications. How often do I stop using the computer for more than a minute? Not very often it seems (is it coffee time yet?).

There is a web-based interface that will let me restore files and would presumably let me download them to wherever I happened to be. It doesn't seem to let you view the files online in your browser which would be very handy, especially for pictures, you can only download the files. Overall it is not as flexible as xdrive promises but it could be handy one day.

Mozy adds a 'Mozy Remote Backup' virtual drive to windows explorer but for me it refuses to open: I click on it but the selection bounces away in buggy fashion. I presume it is supposed to show the folders and files I have backed up and allow me to restore them with drag and drop.

Maybe I should bite the AOL bullet and try xdrive. It would be more flexible even if backup was not totally automatic and didn't offer file versioning. It also gives 5G of space for free but I just cannot help feeling that AOL don't give anything away without a catch. Will they start emailing me .iso disk images?

What I should really be concentrating on is having something like Mozy to backup everyday work in progress and a source control system such as Subversion to handle the versioning of project files. That is the best option. The source control system only holds working code, I don't believe in checking half-finished rubbish in every day just so it is backed up. The backup system is there to save your behind if your hard disk goes down or you accidently delete a file.

Online Backup

Researching online backup solutions: really must backup family photos properly. Options so far:

mozy: 2G free or $4.95 a month unlimited. Automated backup and offers file versioning for a month which would be handy. Files accessable from web.

carbonite: a different shade of mozy, horses for courses. Trial account but no long term free account.

xdrive: 5G free which is enough. Creates a virtual drive which you copy files to however you like or run their backup software. Web based access to files from anywhere, file sharing, lots of nice features. What puts me off? It's owned by AOL

flickr: could just upload multiple G of photos to flickr but since they decided to force me to get a yahoo account I haven't been using flickr (yahoo or AOL, I don't want the hassle). Also the upload wouldn't be as totally automatic as mozy or carbonite.

site5: I could cobble something together on my site5 hosting account: I have about 14G free and ssh access so I should be able to do something but it will be a lashup that I would have to nurse along.

Being a dedicated backup solutions, Mozy or Carbonite would almost force me to do backups and do them well. With anything that looked like another hard disk I would end up with yet another disorganised mess. Less flexibility than xdrive could be a good thing. And by avoiding AOL and yahoo I stick it to "the man".

Yet Another Backup Strategy

Since plugging Cobian Backup last week I have noticed that it leaves two processes running all the time, committing about 15M of memory, 7M of it for a UI to support a status icon, the rest for the backup service. Since the backup only runs once a day for a minute or so I don't see the point of committing resources all the time: the Windows scheduler service is running all the time anyway, so why run another service to do backups?

I've written a new batch file to do a rolling backup using 7zip, a very nice compression package:

rem
rem yet  another backup script
rem
erase "%2\Backup9.7z"
move "%2\Backup8.7z" "%2\Backup9.7z"
move "%2\Backup7.7z" "%2\Backup8.7z"
move "%2\Backup6.7z" "%2\Backup7.7z"
move "%2\Backup5.7z" "%2\Backup6.7z"
move "%2\Backup4.7z" "%2\Backup5.7z"
move "%2\Backup3.7z" "%2\Backup4.7z"
move "%2\Backup2.7z" "%2\Backup3.7z"
move "%2\Backup.7z" "%2\Backup2.7z"

"c:\program files\7-zip\7z" a -r -x@"c:\bin\exclude.txt" "%2\Backup.7z" "%1\*

Exclude.txt is a list of file extensions that should not be backed up.

A wrapper batch file is created to call the above for each backup source:

c:\bin\backup c:\desktop g:\backup\desktop

and this wrapper is run from the Windows scheduler.

I am using 7zip because of the way it allows a source directory to be specified: I'm not sure regular info-zip supports this so cleanly.

7Zip native zip format creates smaller archive files than regular zip (albeit more slowly) and yet salamander still allows me to view them like regular directories so for me there is no practical difference between zip and 7zip. 7zip can be told to create regular zip files is need be.

7Zip supports incremental backups and if my backup files became very large I would consider using it but the external hard disk that I backup to is 80G and has space aplenty so there isn't much to be gained from this.

Cobian backup 7

Found Cobian Backup a nice windows tool for doing automated backups. What is nice about it is that it backs up to zip files so backups are nicely compressed in a non-proprietary format and restoring then does not need any special tools: I can browse the archives from salamander, view them and copy them individually or on mass. It can be set up to do incremental backups every day, a full backup every week and to keep backups of every backup so you have some kind of history.

I am using it simply to archive my work in progress: the computer I am using dies randomly once or twice a week and I don't trust it (it flashes a blue screen and reboots).

I set up the scheduler to do the backup every lunch time, backing up my desktop (where I chuck everything these days) to an external USB hard disk.

The only bugbear I have with it is that it is tedious to get it to exclude files by extension: you have to add them one at a time (click add, enter extension, ok, click add..) and with VC development there are many extensions that can be excluded from backup: my typical list of exclusions is:

*.obj
*.sbr
*.bsc
*.zip
*.map
*.wsp
*.pdb
*.pch
*.tmp
*.res
*.ilk
*.exp
*.idb
*.ncb
*.opt
*.plg

It seems to store it's configurations in C:\Program Files\Cobian Backup 7\DB\MainList.lst which is a text file so I edited it in Vim, rebooted Cobian and job done.

Many quality windows freeware tools listed here and here.

Another good source of free stuff is this.

Unison

I've been plying with unison recently, as a means of replicating my collection of 3000 photos between two computers: I nearly lost all the family photos once before, and I have been neglecting backups. With my new strategy of having the dell D410 fire up in it's docking station every night and doing backups, it can also synchronise the photo collection with the 500m which runs all the time.

The advantage of unison over rsync is that it synchronises a pair of directories, i.e. if you add, edit or delete files on one copy then the other copy will be updated accordingly. This is a bi-directional thing, changes made to the second copy will be replicated in the first. In the case of my photo's, I always use the 500m to copy them from the camera but I am likely to use the D410 to manipulate them and what I want is changes in one archive to be fed into the other.

The invocation of unison that I have come up with so far is:

unison -batch -fastcheck true <path1> <path2>

-batch: tells unison not to ask me stupid questions, it will reconcile problems automatically where it can. -force can be used toi resolve conflicts in favour of one of the paths.
-fastcheck true: tells it to compare files by timestamp and length rather than a full binary compare, as is the default under windows.

I have been using a native windows build of unison rather than the cygwin version, mainly because the windows version handles windows file names transparently, including network file names. cygwin unison via ssh does not seem to be able to find any of the file paths I give it (e.g. /cygdrive/c/blah, the cygwin form of c:\blah) and doesn't like network path names (e.g. \\server\share).

Still, it is not working properly for my photo collection but it may be because the network is not reliable enough to handle 3000 photos and 3G worth of data with absolutely no network problems. This may be another sign that the Wifi on the 500m is iffy (iffy wifi ha!) as the d410 in it's docking station has wired networking available.

Have I mentioned recently that the d410 is really nice? The 500m screen seems huge when I go back to it but the d410 is so fast, small and light.

Dell D410 Docking Station

What with it being the Christmas holidays I found a spare couple of hours to set up the Docking Station for my Dell D410 Laptop. I set it up thusly:

17" CRT monitor
Packard bell keyboard
Packard bell wireless mouse
USB 2 Hub
External USB hard Drive
Wired network so I can play with wake-on-lan and access it from the internet

I need to buy a long audio cable so I can plug it into the hifi.

It makes a nice system, the Dell is faster than the desktop. I still use the desktop system for the following:

Canon USB Scanner
Nightly backup of remote server

I could do my scanning on the laptop and I could set it up to do the nightly server backup as well. Since I set the desktop machine up for nightly backup I haven't got around to getting it to shut down automatically so one of my bedtime tchores is to turn it off. Once that is done I don't need the desktop...

I would like to set the laptop up for a nightly backup to the external hard disk: turn itself on at 3am, backup, then switch itself off. I'd like to figure out the most efficient way to do that: weekly full, and daily incremental maybe. This could be combined with the remote server backup.

The only disadvantage of a laptop over a desktop for me is that it has no PCI slots: the only PCI slot I could use is my TV card and I only need that to capture my wedding video (2 years, 3 months, still haven't got around to doing that). I'm not into games now (did I mention I'm married?) so I'm not inclined to upgrade the graphics: I'd buy an XBox 360 for that.