Copyright (c) <YEAR>, <OWNER> All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of the <ORGANIZATION> nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The backup daemon, bbackupd, runs on all machines to be backed up. The store server daemon, bbstored runs on a central server. Data is sent to the store server, which stores all data on local filesystems, that is, only on local hard drives. Tape or other archive media is not used. The system is designed to be easy to set up and run, and cheap to use. Once set up, there should be no need for user or administrative intervention, apart from usual system maintenance.

bbackupd is configured with a list of directories to back up. It has a lazy approach to backing up data. Every so often, the directories are scanned, and new data is uploaded to the server. This new data must be over a set age before it is uploaded. This prevents rapid revisions of a file resulting in many uploads of the same file in a short period of time. It can also operate in a snapshot mode, which behaves like traditional backup software. When instructed by an external bbackupctl program, it will upload all changed files to the server. The daemon is always running, although sleeping most of the time. In lazy mode, it is completely self contained -- scripts running under cron jobs are not used. The objective is to keep files backed up, not to make snapshots of the filesystem at particular points in time available. If an old version of the file is present on the server, a modified version of the rsync algorithm is used to upload only the changed portions of the file. After a new version is uploaded, the old version is still available (subject to disc space on the server). Similarly, a deleted file is still available. The only limit to their availability is space allocated to this account on the server Future versions will add the ability to mark the current state of files on the server, and restore from this mark. This will emulate the changing of tapes in a tape backup system.

Restoring files is performed using a query tool, bbackupquery. This can be used to restore entire directories, or as an 'FTP-like' tool to list and retrieve individual files. Old versions and deleted files can be retrieved using this tool for as long as they are kept on the server.

bbackupd uses only a minimal amount of disc space to store records on uploaded files -- less than 32 bytes per directory and file over a set size threshold. However, it minimises the amount of queries it must make to the server by storing, in memory, a data structure which allows it to determine what data is new. It does not need to store a record of all files, essentially just the directory names and last modification times. This is not a huge amount of memory. If there are no changes to the directories, then the client will not even connect to the server.

Box Backup is designed to be secure in several ways. The data stored on the backup store server is encrypted using secret-key cryptography. Additionally, the transport layer is encrypted using TLS, to ensure that the communications can't be snooped.

The files, directories, filenames and file attributes are all encrypted. By examining the stored files on the server, it is only possible to determine the approximate sizes of a files and the tree structure of the disc (not names, just number of files and subdirectories in a directory). By monitoring the actions performed by a client, it is possible to determine the frequency and approximate scope of changes to files and directories. The connections between the server and client are encrypted using TLS (latest version of SSL). Traffic analysis is possible to some degree, but limited in usefulness. An attacker will not be able to recover the backed up data without the encryption keys. Of course, you won't be able to recover your files without the keys either, so you must make a conventional, secure, backup of these keys.

SSL certificates are used to authenticate clients. UNIX user accounts are not used to minimise the dependence on the configuration of the operating system hosting the server. A script is provided to run the necessary certification authority with minimal effort.

The server daemon is designed to be simple to deploy, and run on the cheapest hardware possible. To avoid the necessity to use expensive hardware RAID or software RAID with complex setup, it (optionally) stores files using RAID techniques. It does not need to run as a privileged user. Each account has a set amount of disc space allocated, with a soft and a hard limit. If the account exceeds the soft limit, a housekeeping process will start deleting old versions and deleted files to reduce the space used to below the soft limit. If the backup client attempts to upload a file which causes the store to exceed the hard limit, the upload will be refused.

Firstly, check that all the clocks on your clients, servers and signing machines are accurate and in sync. A disagreement in time between a client and a server is the biggest cause of installation difficulties, as the times in the generated certificates will cause login failures if the start date is in the future.

In the following instructions, replace 0.00 with the actual version number of the archive you have downloaded. For help building on Windows, see the Windows Compile Appendix. And if you want to build a Linux RPM, look here. You need the latest version of OpenSSL, as some of the extra APIs it provides are required. You should have this anyway, as earlier versions have security flaws. (If you have an earlier version installed, the configuration script will give you instructions on enabling experimental support for older versions.) See OpenSSL notes for more information on OpenSSL issues. There are some notes in the archive about compiling on various platforms within the boxbackup-0.00 directory -- read them first. For example, if you are compiling under Linux, look for LINUX.txt as boxbackup-0.00/LINUX.txt after untaring the archive. Download the archive, then in that directory type tar xvzf boxbackup-0.00.tgz cd boxbackup-0.00 ./configure make The server and client will be built and packaged up for installation on this machine, or ready to be transferred as tar files to another machine for installation. This builds two parcels of binaries and scripts, 'backup-client' and 'backup-server'. The generated installation scripts assumes you want everything installed in /usr/local/bin Optionally, type make test to run all the tests.

Type make install-backup-client to install the backup client. Type make install-backup-server to install the backup server.

In the parcels directory, there are tar files for each parcel. The name reflects the version and platform you have built it for. Transfer this tar file to the remote server, and unpack it, then run the install script. For example: tar xvzf boxbackup-0.00-backup-server-OpenBSD.tgz cd boxbackup-0.00-backup-server-OpenBSD ./install-backup-server

You can use arguments to the configure script to adjust the compile and link lines in the generated Makefiles, should this be necessary for your platform. The configure script takes the usual GNU autoconf arguments, a full list of which can be obtained with --help. Additional options for Box Backup include: --enable-gnu-readline Use GNU readline if present. Linking Box Backup against GNU readline may create licence implications if you then distribute the binaries. libeditline is also supported as a safe alternative, and is used by default if available. --disable-largefile Omit support for large files --with-bdb-dir=DIR Specify Berkeley DB location --with-random=FILE Use FILE as random number seed (normally auto-detected) --with-tmp-dir=DIR Directory for temporary files (normally /tmp) See OpenSSL notes for the OpenSSL specific options.

There are a number of unit tests provided. To compile and run one type: ./runtest.pl bbackupd release ./runtest.pl common debug ./runtest.pl ALL The runtest.pl script will compile and run the test. The first argument is the test name, and the second the type of build. Use ALL as a test name to run all the tests. The output from the tests is slightly muddled using this script. If you're developing, porting or trying out new things, it might be better to use the following scheme: cd test/bbackupd make cd ../../debug/test/bbackupd ./t or in release mode... cd test/bbackupd make -D RELEASE cd ../../release/test/bbackupd ./t (use RELEASE=1 with GNU make) I tend to use two windows, one for compilation, and one for running tests.

Ideally, you need to use version 0.9.7 or later of OpenSSL. If this is installed on your system by default (and it is on most recent releases of UNIX like OSes) then everything should just work. However, if it isn't, you have a few options.

The best option is to upgrade your installation to use 0.9.7. Hopefully your package manager will make this easy for you. This may require reinstallation of lots of software which depends on OpenSSL, so may not be ideal. (But as there have been a few security flaws in OpenSSL recently, you probably want to upgrade it anyway.)

The second best option is to install another copy. If you download and install from source, it will probably install into /usr/local/ssl. You can then configure Box Backup to use it using: ./configure --with-ssl-headers=/usr/local/ssl/include --with-ssl-lib=/usr/local/ssl/lib which will set up the various includes and libraries for you. The configuration scripts may be a problem, depending on your installation. See below for more information.

If you have an old version installed, the configuration script will give you instructions on how to enable support for older versions. Read the warnings, and please, whatever you do, don't release binary packages or ports which enable this option. You may have issues with the configuration scripts, see below.

If you get OpenSSL related errors with the configuration scripts, there are two things to check: The bin directory within your OpenSSL directory is in the path (if you have installed another version) You have an openssl.cnf file which works and can be found.

You need to have an openssl.cnf file. The default will generally work well (see example at end). Make sure the openssl utility can find it, either set the OPENSSL_CONF environment variable, or install it into the location that is mentioned in the error messages. Example OpenSSL config file: # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # RANDFILE = /dev/arandom #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes [ req_distinguished_name ] countryName = Country Name (2 letter code) #countryName_default = AU countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) #stateOrProvinceName_default = Some-State localityName = Locality Name (eg, city) 0.organizationName = Organization Name (eg, company) #0.organizationName_default = Internet Widgits Pty Ltd # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = CryptSoft Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg, fully qualified host name) commonName_max = 64 emailAddress = Email Address emailAddress_max = 64 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ x509v3_extensions ] nsCaRevocationUrl = http://www.cryptsoft.com/ca-crl.pem nsComment = "This is a comment" # under ASN.1, the 0 bit would be encoded as 80 nsCertType = 0x40

This Appendix explains how to build the bbackupd daemon for Windows using the Visual C++ compiler. If you have any problems following these instructions, please sign up to the mailing list and report them to us, or send an email to Chris Wilson. Thanks! Note: bbstored will not be built with this process. bbstored is not currently supported on Windows. There are no plans for bbstored support on Windows.

You will need quite a bit of software to make this work. All of it is available for free on the Internet, although Visual C++ Express has license restrictions and a time limit.

Microsoft's Visual C++ compiler and development environment are part of their commercial product Visual Studio. Visual Studio 2005 is supported, and 2003 should work as well. You can also download a free copy of Visual C++ 2005 Express. This copy must be registered (activated) within 30 days, and is free for one year. You will need the Platform SDK to allow you to compile Windows applications.

Download and install ActivePerl for Windows, which you can probably find here.

You will need to download and install several libraries. They must all be built in the same directory, to be able to link properly. Choose a directory where you will unpack and compile OpenSSL, Zlib and Box Backup. We will call this the base directory. An example might be: C:\Documents and Settings\Your Username\Desktop\Box Make sure you know the full path to this directory.

You will need to compile OpenSSL using Visual C++. The latest release at this time, OpenSSL 0.9.8a, does not compile with Visual C++ 2005 out of the box, so you need a patched version. The original source and patch are also available. To compile OpenSSL: Use a Windows unzipper such as WinZip (free trial) to extract the openssl-0.9.8a-vc2005.tar.gz archive, which you just downloaded, into the base directory. Rename the folder from openssl-0.9.8a-vc2005 to openssl Open a command shell (run cmd.exe) and cd to the openssl directory Run the following commands: perl Configure VC-WIN32 ms\do_ms "c:\program files\Microsoft Visual Studio 8\Common7\Tools\vsvars32.bat" nmake -f ms\ntdll.mak

You will need to download the Zlib compiled DLL. Create a directory called zlib in the base directory, and unzip the file you just downloaded into that directory. You don't need to compile anything.

The first version of Box Backup that's known to compile and with Visual C++ 2005 is available on the Subversion server. However, this version has not been extensively tested and may be out of date. The changes are expected to be merged into the Subversion trunk at some point, and this page should then be updated. If in doubt, please sign up to the mailing list and ask us. To get the source code out of Subversion you will need a Subversion client for Windows. After installing it, open a new command prompt, go to the base directory, and type: svn co http://bbdev.fluffy.co.uk/svn/box/chris/win32/vc2005-compile-fixes/ boxbackup This should create a directory called boxbackup inside the base directory.

Open a command prompt, change to the base directory then boxbackup, and run win32.bat to configure the sources. Otherwise, Visual C++ will complain about missing files whose names start with autogen, and missing config.h.

Open Visual C++. Choose "File/Open/Project", navigate to the base directory, then to boxbackup\infrastructure\msvc\2005 (or 2003 if using Visual Studio 2003), and open any project or solution file in that directory. Press F7 to compile Box Backup. If the compilation is successful, boxbackup\Debug\bbackupd.exe will be created.

Create the destination directory, C:\Program Files\Box Backup\bbackupd. Write a configuration file, keys and certificate on a Unix machine, and copy them into the Box Backup directory, together with the following files from the base directory: boxbackup\Debug\bbackupd.exe openssl\out32dll\libeay32.dll openssl\out32dll\ssleay32.dll zlib\zlib1.dll Ensure that the user running Box Backup can read from the Box Backup directory, and write to the bbackupd directory inside it. Run Box Backup by double-clicking on it, and check that it connects to the server. If the window opens and closes immediately, it's probably due to a problem with the configuration file - check the Windows Event Viewer for details.

Box Backup can also run as a Windows service, in which case it will be automatically started at boot time in the background. To install this, open a command prompt, and run: cd "C:\Program Files\Box Backup" bbackupd.exe -i This should output Box Backup service installed.

It is very easy to build an RPM of Box Backup on Linux platforms. This should work on all Red Hat distributions (including Fedora), SuSE, and probably others too. Given that you have the correct development packages installed simply execute this command (replacing the version number): rpmbuild -ta boxbackup-0.00.tgz rpmbuild will report where the packages have been written to, and these can be installed in the normal manner. If you have never built an RPM before you should set up a convenient build area as described in the RPM book. If you wish to customise the package you can find the spec file in the contrib/rpm directory.