Back In Time: Linux Backup Made Easy (Part 1)

demo

Linux is a bit of a funny creature.  In many ways, it’s the most integrated and full featured operating system in existence.  Take software development, for example.  You couldn’t ask for a more integrated, well thought out solution.  The tools are absolutely first rate, the system is stable, and because the platform is open source, most of the rough edges get worn off rather quickly.But there’s also a downside to open source.  You see, everyone tends to scratch their own itch first.  Thus, while there are a bevy of tools that perfectly meet the needs of a few (most notably their authors), they often miss the needs of the many.  That is a problem, because the masses don’t necessarily have the skill set required to adapt an already existing program so that it fits them.  If anything, this is why Microsoft is valuable.  They create software that more or less handles the needs of the masses.
And as might be expected, this is also the state of backup on Linux.  It is no understatement to say that you can get your hands on the very best backup tools available, for free.  All it requires is that you use some derivative of the following:

1. sudo apt-get install <em>toolname</em>

复制代码

Unfortunately, while you can find the very best tools, that doesn’t necessarily mean that you will be able to use them.  Rather that be written with the end-user in mind, they were created for programmers and system administrators.  You know, the type of people who wonder why anyone would bother with a user interface.  After all, it’s so much faster to do everything from the command line.
The result is that while the Linux backup tools may be powerful, they also have a learning curve steep enough to bang your head against.  And that is an absolute shame.  In general, I (and a great deal of humanity in general) am against activities that require me to learn new things or otherwise grow in what might end up being a painful direction.  Even if it is for a good cause.   (And I can think of few more important than making sure thatmy personal information is safe and secure.)  Ideally, I want backup to be easy.  I  want to “Set it and forget it.”  Time Machine for Mac OS X allows me to do this, as does the Windows Vista File Backup.  But at first pass, such a solution for Linux is woefully absent.

That’s not to say that there aren’t contenders.  In fact, a quick search on Google will reveal several: sbackup, Flyback, and TimeVault all look promising.  But if you look at their project pages, you will notice something disconcerting: in each case, the programs are old and there isn’t much recent activity.  This is typically a bad sign for an open source project.  No activity usually means that the program is dead and unlikely to advance further.  And that is really too bad, because sbackup, Flyback and TimeVault are good programs with potential.  But, like many other open source options, they have a number of very rough edges.

Maybe that’s why I got so excited when I saw a new contender, called “Back In Time,” when it was recently featured on Lifehacker.  At the time, I thought, “Have I finally found a backup program I can really use?”  Now, after four months of using it full time, I can answer that question: Yes!  Without a doubt!

Back In Time

Back in time is the full package.  It includes a robust backend that makes incremental snapshots easy.  You’ll also find a simple to use restoration system and a settings pane that actually make sense.  What’s more, the defaults are actually useful!  As a particularly nice touch, there are even separate packages for KDE and Gnome, so that the program will natively blend with whatever desktop you happen to have installed.

Installation

On Ubuntu, you can install Back In Time by downloading the packages for your Linux distribution.  As noted above, you can find two different user interfaces available: one for KDE and another for Gnome.  Don’t fret about downloading one over the other.  They are identical in terms of functionality.  (Note: The screenshots you see in this article were taken from the Gnome user interface.)
Configuration

Thanks to the well-thought out defaults, getting started with Back In Time is easy and straightforward.  The first time you launch the program, you are prompted to configure the differenct backup options.  (Should you ever want to change these, you can access the same pane by clicking on the “Settings” button.)  In the configuration window, you will see several different tabs.  Each tab corresponds to a different part of the backup job.  There are the general settings which describe where you want your backups to be saved and how often you want the backup to run.  And there are settings for what you want to include, exclude, and how many snapshots you want to keep.  Back In Time can back up to any attached storage device, including a USB hard drive or flash drive.  Unfortunately, however, it doesn’t work with network drives or with optical media.

The include tab lets you specify which folders you would like to have added to the backup.  Since Back In Time runs in two different modes (user and root), I’ve been using each mode to accomplish a slightly different goal.  The user mode is ideal for backing up files and settings, while the root mode is ideal for backing up the system itself (similar to the way in Time Machine works on Mac OS X).

By using the different modes in this way, I can keep user fuke backups are kept in one location and the system information in another.  Thus, if I should lose my system, I can quickly and easily restore my system using the "System" backup and then copy back my files.  I avoid duplicating data through the “Exclude” tab.  In the root mode, I exclude the “home” directory, where all of the files  and profiles are kept.  But since I am backing up my home directory through the user mode, this doesn’t really matter.

Incremental backups are most valuable when run frequently.  Time Machine creates snapshots every hour for more or less “maximum” protection.  And as I learned recently (after accidently deleting my entire source code directory for LyX-Outline), hourly backups can be a tremendous lifesaver.  After my "accident," I was able to restore the directory to the way it had looked just 15 or so minutes before.  In doing so, I recovered a number of important changes that hadn’t yet been committed to Subversion.  At that moment, I was tremendously grateful that Back In Time could be configured for hourly snapshots.

But over time, the side effect of hourly snapshots is a huge number of unnecessary backups clog the drive.  After all, I don't need to know what my drive looked like during every hour of 2009.  It's here that we see another fine example of Back In Time's overall polish.  The auto-remove settings provide a way to quickly and efficiently deal with too many snapshots.  Back in time can be configured to prune the number of backups to something that is slightly more reasonable.

Restoring Files

File restoration is dead simple as well.  You can quickly browse through backed up files and folders using the restore pane of the GUI.  If you need a version from an older snapshot, simply select it from the “snapshots” menu on the left.  Once you have selected the files you are interested in, click “Restore.”  They will be copied to their old locations.

demo

Limitations

But even though Back In Time is probably the best user-oriented backup solution I’ve found for Linux, it still isn’t perfect.  (Though really, really close.)  There is at least one major limitation: lack of a network based backup.
As I’ve said before, I prefer to keep my backups on my home server.  There are a number of reasons for this, but the most important is that I don’t need to remember to turn my backup drive on.  When it’s time to automatically back up, it just happens.  Unfortunately, Back In Time does not natively support network or remote backups.  (Fortunanately it runs on Linux, the most customizable operating system in the world, and there happens to be a workaround.)

Conclusion

Back In Time is a wonderful piece of software.  In fact, it’s the only backup program on Linux I’ve found that really meets the dead simple needs of the typical computer user.  It is built on the solid foundation of rsync, allows for incremental backups, has a well though out GUI and defaults, and generally “Just Works © .”
Even its greatest weakness (the lack of a network based backup option) is more of an inconvenience than a truly show stopping fault, since the same thing can readily be accomplished using a Linux kernel extension and a simple script.  In Part 2 of this article, I will show you how.

demo

Despite all of its strengths, Back In Time has one major weakness: it doesn’t support backup over a network.  But luckily, it runs on Linux,the single most customizable operating system in the world!  As a result even though Back In Time might not officially support a backup to a remote computer, you can still make it work through a little bit of spit and grit.The magical sauce is the Fuse SSH file system.  This kernel module allows for Linux to mount a remote share as a local drive.  And by doing so, Back In Time thinks that it is working with a local folder even though the data might be sent across the network or internet.  To make it work, though, you’ll have to work at the command line and do a bit of programming.  But, don’t panic because it isn’t that hard.  It amounts to a few shell commands and about six lines of code.  Below, I’ll show you how in seven simple steps.
Step 1: Prepare a Local Mount Point and Set Permissions
First, we first need to create a mount point.  A mount point is a blank folder that the operating system uses to connect the local computer to the remote server.  I typically like to have my mount points in the /media folder.  If the mount point is in the /media folder, it will show a little icon on the desktop when active.
So, open up the terminal and type the following:

1. sudo mkdir /media/Backup

复制代码

This command will create a new folder (Backup) in the media folder.  As is, we could then mount the SSH file system to this mount point, but to do so would require that we run the command as the super user (root).  It’s not necessarily good to run things as root.  So, let’s change the folder’s owner and permissions so that we can mount it without requiring the super user:

1. sudo chown –R <em><Username></em> /media/Backup
   
2. 
   
3. sudo chmod a+x /media/Backup

复制代码

In the first command above, replace <Username> with the username that you use to log-on to Linux.  In my case, it would be: roakes.  The second command properly sets the permissions so when it comes time to mount the remote share, the computer won’t prompt for the root password.


Step 2: Install the SSH File System (SSHFS)
Now that we have a valid mount point, let’s install the software packages for SSHFS.  If using Ubuntu, type the following into your system terminal:

1. sudo apt-get install sshfs

复制代码

Step 3: Get the IP Address or Url of the Remote Computer

With the mount point set and software installed, we need to do a bit of planning.  “Proper preparation prevents pitifully poor performance” and all that.  First, we need to know where the data will be stored.  Will we be using a simple home server or will we be storing that data on a remote server over the internet?  Also, will you be able to connect to the server using the secure shell (SSH) protocol?  If so, what is the destination folder?
We will use this information to construct a connection string.  It will have the form:

1. <Username>@Machine.Location:/path/to/backup/folder

复制代码

SSH to a Machine on the Local Network
If we will be backing up to a computer on the local network, we need to know it’s IP address so that we can establish an SSH connection.  To find this, go to the server and log-on.  Then type ifconfig into the terminal window.  This will return a block of text similar to that seen below, which was copied from my server.  I have added a series of x to some of the information.

eth0 Link encap:Ethernet HWaddr 00:xx:xx:xx:e0:93
inet addr:xxx.xxx.1.106 Bcast:xxx.xxx.1.255 Mask:255.255.255.0
inet6 addr: fe80::xxx:xxxx:xxxx:xxxx/64 Scopeink
UP BROADCAST RUNNING MULTICAST xxx:1500 Metric:1
RX packets:26460698 errors:23 dropped:145 overruns:17 frame:0
TX packets:14444204 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2320464437 (2.1 GB) TX bytes:2079312151 (1.9 GB)
Interrupt:3 Base address:0xd800
lo Link encapocal Loopback
inet addr:127.0.0.x Mask:255.0.0.0
inet6 addr: ::x/xxx Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:9308 errors:0 dropped:0 overruns:0 frame:0
TX packets:9308 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4232264 (4.0 MB) TX bytes:4232264 (4.0 MB)

The IP address can be found on the second line below eth0, which starts : inet addr.  In my case, the IP address is: xxx.xxx.1.106.  The xxx.xxx will have numeric values that are specific to your network.  To connect to the machine, I use a connection string that looks like:

1. <em><Username></em>@xxx.xxx.1.106:/path/to/backup/folder

复制代码

SSH to a Machine on the Internet
If you will be connecting to another computer on the internet, then instead of using an IP address to connect, you will instead be using a url.  You can get this information from the information page of your backup service, or from the network system administrator.  To connect to the remote machine, you will use a connection string that looks like:

1. <em><Username></em>@<em>machine.url:/path/to/backup/folder</em>

复制代码

To remotely access my outside of the network computer, I use:

1. roakes@remote.oak-tree.us:/media/Linux-Backup

复制代码

Step 4: Test the Connection
Once I have the connection string, I can now test SSHFS and make sure that it can mount to my mount point (Step 1).  To mount the remote drive, type:

1. sshfs <em><Username>@Machine.Location</em>:<em>/path/to/backup/folder</em> /media/Backup

复制代码

Be sure to replace the part in italics with your connection string.  The first time that you connect, you will be prompted to accept the server’s key and be asked for your password.  After you enter your user credentials, it should mount the file system to /media/Backup.  Open up the location in finder and make sure that you can create folders and move files onto the remote machine.
You can unmount the volume by typing (no root required):

1. fusermount –u /media/Backup

复制代码

Step 5: Use a Passkey Instead of a Password
While a password is acceptable when you want to connect manually, we want the backup script to run automatically.  That means no password prompt.
Luckily, however, it’s easy to configure Linux so that it uses a passkey instead of a password.  This will allow your computer to connect automatically instead of prompting you for your user credentials each time that you want to run a backup or restore a file.  To do so:

• Generate a key on your computer by typing ssh-keygen at the terminal.  You will be prompted for the file where you want to save the key.  Just use the default (/home/username/.ssh/id_rsa).  It will then ask for a passphrase, leave this empty.
• Next, append the public key that you just created to the authorized keys file on the remote server.  To do so, type:
  cat ~/.ssh/id_rsa.pub | ssh <Username>@Machine.Location ‘cat >> .ssh/authorized_keys'
  

Now when you use the command from Step 4 above, it will automatically mount without prompting for a password.

demo

Step 6: Put It All Together With Custom Scripts

Now that we have the connection string and have configured a passkey instead of a password, we are ready put all the pieces together in two custom scripts.  The first will be used for automating the backups and the second will be used for opening the GUI.  There are three things in particular that need to happen:

• The computer mounts the remote drive to the appropriate point in the file system.
• Either the user interface or automated backup routine runs.
• The remote store is unmounted when the program finishes and exits.
  

Creating an Automatic Backup
Below is a template for a script that can be used to start a backup job:

sshfs <Username>@Machine.Location:/path/to/backup/folder /media/Backup
backintime -b
fusermount -u /media/Backup

The first line mounts the remote folder, the second launches the command line version of backintime and tells it to make a new snapshot, and the last line then unmounts the remote drive after Back In Time has finished.  Save this file to someplace in your home file as “Data-Backup.sh”.  I like to keep all of my scripts together in a folder imaginatively called “Applications.”
Anytime that you want to run an automated backup, you can do so by typing:

bash /path/to/folder/Backup-Data.sh

Launching the GUI
Now that we have a script for running the backup to the remote location, we are half way done.  But as a backup is only as good as ability to access the data, we also need a second script.  This will be used for launching the program, retrieving files, or making changes to the configuration.  Again, we follow the same steps as the automatic backup script.  Below is the template:

sshfs <Username>@Machine.Location:/media/backup/Backup/Linux-Backup /media/Backup
backintime-gnome
fusermount -u /media/Backup

The first line mounts the remote share, the second line launches the GUI version of Back In time, and the third unmounts the volume after you exit.  Save this as “GUI-Back-In-Time.sh” in your scripts folder.
Step 7: Easy Access and Automation
With your two custom scripts, you have all of the tools needed to backup and restore files to your remote computer.  But launching the terminal every time you need to do one of these jobs can be a bit of a pain.  And because we are backing up to a remote location, we can’t use the scheduler built in to Back In Time.  So, to make things easier, let’s:

• Create a custom menu launcher for easy access
• Automate the backup process using Gnome Schedule and the user crontab file.
  

Create a Custom Menu Launcher for Easy Access
To make it easier to access our custom script, let’s create a custom launcher by using the Applications->System Tools->Main Menu utility.

After the utility launches, click on the “System Tools” sub-menu and then push the “New Item” button.  In the pop-up, specify the type of launcher as “Application” and the name as “Back In Time.”  For the command, use:

bash /path/to/scripts/folder/GUI-Back-In-Time.sh

Add a description to remind you what Back In Time does and, if you want, you can  customize the icon by clicking on the picture at left.  When finished, press “Close.”  This will add your custom launcher to the main menu.

Automated Backup: Cron Jobs and Gnome ScheduleAfter all that preliminary work, we’re finally able to automate our backup.  We’ll accomplish this through the use of the Linux cron daemon.  Cron is a program that allows for tasks to be executed at a certain time.  You can create cron tasks to update a website, or automatically download files, or perform system maintainence tasks.
Cron tasks are specified in a file called the “crontab.”  Each user has a crontab file.  There is also a system crontab that runs under the root user.
While editing the cron file by hand is easy enough, we will be using a special GUI called “Gnome Schedule.”  To install it, type the following command in the terminal:

sudo apt-get install gnome-schedule

After the installation finishes, you can launch the utility by going to Applications->System Tools->Scheduled Tasks.
To create a new scheduled task, click on the “New” button and then choose “Recurrent Task” from the drop down list.  A recurrent task is run at a specified interval: every second, minute, hour, month, or year.  If you want to be fancy, you can specify which hour of the day that you would like it execute.
For the command, use the following:

bash /path/to/scripts/folder/Data-Backup.sh

When finished, press “Add.”

Conclusion
Congratulations!  Through the use of the sshfs kernel module, you've now configured Back In Time to more or less duplicate all of the behavior of Time Machine.  We've fooled it into thinking that it's working on the local drive, when in reality that archive may be sitting across the network or across the world.  And though the instructions provided here were specific to the SSH file system extension, extensions can also be found for Amazon Simple Storage Solutions (S3), WebDav, FTP and Samba.  This makes it possible to backup your data to almost any service or destination imaginable.  Even better, through the use of scripting, it is possible to automate the process so that it reaches “set it and forget it” simplicity.

Oak Tree