5. Backups

This time I’m not taking any chances:  I’m going to have a backup system in place from the get-go.  Backups use duplicity and are sent offsite to Amazon S3 storage, which is surprisingly affordable and really secure.  Most of the setup process is nicely covered by this tutorial, but I’m using my own backup script.  Credit for the original shellscript on which this was based goes to Daniel Wallin. 

Why a custom script?

  1. I have to do more complicated things, like taking LVM (and eventually ZFS) snapshots to back up from
  2. I needed to be able to save and reconstruct hard links in the filesystem.  That functionality isn’t supported by duplicity but fortunately Daniel had figured out how to handle it.
  3. This job is just way too important to entrust to a shell script, especially one that doesn’t use “set -e”.  Errors tend to hide and the script barrels ahead regardless of whether everything’s okay or not.  Originally boostpro.com was doing this in bash, but eventually the complexity got out of hand and we’re transitioning to Python.

The one thing I had to remember to do that isn’t covered by the tutorial was to install nullmailer so I could get logs from the cron job. If you dpkg-reconfigure nullmailer you can tell it to send all emails to users on that server somewhere else; I just gave it my normal email address.

Installation

Download the duplicity sources. As of this writing, the latest is 5.09

On Debian/Ubuntu:

$ tar xvf duplicity-*.tar.gz
$ cd duplicity-*/
$ sudo apt-get build-dep duplicity
$ sudo aptitude install python-boto checkinstall
$ sudo checkinstall python setup.py install
$ cd ..

Hit return a few times, and duplicity will have been installed.

Note

I had a little trouble with earlier versions of duplicity.
I’m not sure whether that was my fault, but the latest seems to be
the greatest, too.

Now either click the download button on the repository’s home page and unpack the result, or

$ git-clone git://github.com/techarcana/s3backup.git

then:

$ sudo chown -R root:root s3backup
$ sudo mv s3backup/etc/cron.d/* /etc/cron.d
$ sudo cp s3backup/aws_secrets.py.sample s3backup/aws_secrets.py
$ sudo mv s3backup /usr/local
$ sudo chmod 0600 /usr/local/s3backup/aws_secrets.py

now edit your private information into /usr/local/s3backup/aws_secrets.py.

And that should do it!

Final Steps

Be sure to tell updatedb not to index /usr/local/s3backup, or it will go pawing through all your mounted snapshots! In /etc/updatedb.conf:

PRUNEPATHS="/tmp /var/spool /media /usr/local/s3backup"

Troubleshooting

My experience with duplicity has been replete with gotchas and bad error messages that made it hard to diagnose dumb mistakes. Here are some things to watch out for right off-the-bat:

  1. Be sure to verify that you can restore files from these backups! We don’t have an automated restore procedure yet, but if you pass -v to backup.py you’ll get enough information about the duplicity calls to understand what to do.
  2. You may find that you need to sign or trust your gpg key per notes in this bug. I did, in order to be able to look at the backups.
  3. If you use sudo without the -H option to invoke duplicity manually, it doesn’t set the $HOME environment variable to the target user’s (e.g. root’s) home directory. That will make gpg look in the wrong place for its key files, with horribly uninformative results. That’s why the backup script explicitly sets $HOME. Remember to use sudo -H to get a reliable environment for gpg.
  4. Another way to get really bad error messages from duplicity is to specify the wrong –archive-dir option. I’ve opened a bug about that with the duplicity devs.
  5. Be sure to store a hard copy of your GPG passphrase in a safe place (or two)! If you lose this information, your backups are wasted.

3 responses

16 02 2009
Phew « TechArcana

[…] 4. Backups Associated Sites […]

14 01 2010
JHF

hi,

can you give more info about the handling of hardlinks within duplicity ?

Thanks

14 01 2010
Dave Abrahams

Sure thing! Take a look at backup.py in this project. In particular, this line which makes a record of hard links in the system and this loop that makes sure that only one reference to any given inode is followed.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: