Box Backup Build and Installation GuideLicenseCopyright (c) <YEAR>, <OWNER>All rights reserved.Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.Neither the name of the <ORGANIZATION> nor the names of
its contributors may be used to endorse or promote products derived
from this software without specific prior written permissionTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.IntroductionThe backup daemon, bbackupd, runs on all machines to be backed up.
The store server daemon, bbstored runs on a central server. Data is sent
to the store server, which stores all data on local filesystems, that is,
only on local hard drives. Tape or other archive media is not used.The system is designed to be easy to set up and run, and cheap to
use. Once set up, there should be no need for user or administrative
intervention, apart from usual system maintenance.Client daemonbbackupd is configured with a list of directories to back up. It
has a lazy approach to backing up data. Every so often, the directories
are scanned, and new data is uploaded to the server. This new data must
be over a set age before it is uploaded. This prevents rapid revisions
of a file resulting in many uploads of the same file in a short period
of time.It can also operate in a snapshot mode, which behaves like
traditional backup software. When instructed by an external bbackupctl
program, it will upload all changed files to the server.The daemon is always running, although sleeping most of the time.
In lazy mode, it is completely self contained -- scripts running under
cron jobs are not used. The objective is to keep files backed up, not to
make snapshots of the filesystem at particular points in time
available.If an old version of the file is present on the server, a modified
version of the rsync algorithm is used to upload only the changed
portions of the file.After a new version is uploaded, the old version is still
available (subject to disc space on the server). Similarly, a deleted
file is still available. The only limit to their availability is space
allocated to this account on the serverFuture versions will add the ability to mark the current state of
files on the server, and restore from this mark. This will emulate the
changing of tapes in a tape backup system.RestorationRestoring files is performed using a query tool, bbackupquery.
This can be used to restore entire directories, or as an 'FTP-like'
tool to list and retrieve individual files. Old versions and deleted
files can be retrieved using this tool for as long as they are kept on
the server.Client Resource Usagebbackupd uses only a minimal amount of disc space to store
records on uploaded files -- less than 32 bytes per directory and file
over a set size threshold. However, it minimises the amount of queries
it must make to the server by storing, in memory, a data structure
which allows it to determine what data is new. It does not need to
store a record of all files, essentially just the directory names and
last modification times. This is not a huge amount of memory.If there are no changes to the directories, then the client will
not even connect to the server.SecurityBox Backup is designed to be secure in several ways. The data
stored on the backup store server is encrypted using secret-key
cryptography. Additionally, the transport layer is encrypted using TLS,
to ensure that the communications can't be snooped.EncryptionThe files, directories, filenames and file attributes are all
encrypted. By examining the stored files on the server, it is only
possible to determine the approximate sizes of a files and the tree
structure of the disc (not names, just number of files and
subdirectories in a directory). By monitoring the actions performed by
a client, it is possible to determine the frequency and approximate
scope of changes to files and directories.The connections between the server and client are encrypted
using TLS (latest version of SSL). Traffic analysis is possible to
some degree, but limited in usefulness.An attacker will not be able to recover the backed up data
without the encryption keys. Of course, you won't be able to recover
your files without the keys either, so you must make a conventional,
secure, backup of these keys.AuthenticationSSL certificates are used to authenticate clients. UNIX user
accounts are not used to minimise the dependence on the configuration
of the operating system hosting the server.A script is provided to run the necessary certification
authority with minimal effort.Server daemonThe server daemon is designed to be simple to deploy, and run on
the cheapest hardware possible. To avoid the necessity to use expensive
hardware RAID or software RAID with complex setup, it (optionally)
stores files using RAID techniques.It does not need to run as a privileged user.Each account has a set amount of disc space allocated, with a soft
and a hard limit. If the account exceeds the soft limit, a housekeeping
process will start deleting old versions and deleted files to reduce the
space used to below the soft limit. If the backup client attempts to
upload a file which causes the store to exceed the hard limit, the
upload will be refused.Building and installingBefore you startFirstly, check that all the clocks on your clients, servers and
signing machines are accurate and in sync. A disagreement in time
between a client and a server is the biggest cause of installation
difficulties, as the times in the generated certificates will cause
login failures if the start date is in the future.Box Backup compileIn the following instructions, replace 0.00 with the actual
version number of the archive you have downloaded.For help building on Windows, see the Windows
Compile Appendix. And if you want to build a Linux RPM, look here.You need the latest version of OpenSSL, as some of the extra APIs
it provides are required. You should have this anyway, as earlier
versions have security flaws. (If you have an earlier version installed,
the configuration script will give you instructions on enabling
experimental support for older versions.)See OpenSSL notes for more information
on OpenSSL issues.There are some notes in the archive about compiling on various
platforms within the boxbackup-0.00 directory -- read them first. For
example, if you are compiling under Linux, look for LINUX.txt as
boxbackup-0.00/LINUX.txt after untaring the archive.Download the archive, then in that directory typetar xvzf boxbackup-0.00.tgz
cd boxbackup-0.00
./configure
makeThe server and client will be built and packaged up for
installation on this machine, or ready to be transferred as tar files to
another machine for installation.This builds two parcels of binaries and scripts, 'backup-client'
and 'backup-server'. The generated installation scripts assumes you want
everything installed in /usr/local/binOptionally, type make test to run
all the tests.Local installationType make install-backup-client
to install the backup client.Type make install-backup-server
to install the backup server.Remote installationIn the parcels directory, there are tar files for each parcel. The
name reflects the version and platform you have built it for.Transfer this tar file to the remote server, and unpack it, then
run the install script. For example:tar xvzf boxbackup-0.00-backup-server-OpenBSD.tgz
cd boxbackup-0.00-backup-server-OpenBSD
./install-backup-serverConfigure optionsYou can use arguments to the configure script to adjust the
compile and link lines in the generated Makefiles, should this be
necessary for your platform. The configure script takes the usual GNU
autoconf arguments, a full list of which can be obtained with --help. Additional options for Box Backup
include:--enable-gnu-readlineUse GNU readline if present. Linking Box Backup against
GNU readline may create licence implications if you then
distribute the binaries. libeditline is also supported as a safe
alternative, and is used by default if available.--disable-largefileOmit support for large files--with-bdb-dir=DIRSpecify Berkeley DB location--with-random=FILEUse FILE as random number seed (normally
auto-detected)--with-tmp-dir=DIRDirectory for temporary files (normally /tmp)See OpenSSL notes for the OpenSSL
specific options.TestsThere are a number of unit tests provided. To compile and run one
type:./runtest.pl bbackupd release
./runtest.pl common debug
./runtest.pl ALLThe runtest.pl script will compile and run the test. The first
argument is the test name, and the second the type of build. Use ALL as
a test name to run all the tests.The output from the tests is slightly muddled using this script.
If you're developing, porting or trying out new things, it might be
better to use the following scheme:cd test/bbackupd
make
cd ../../debug/test/bbackupd
./tor in release mode...cd test/bbackupd
make -D RELEASE
cd ../../release/test/bbackupd
./t(use RELEASE=1 with GNU make)I tend to use two windows, one for compilation, and one for
running tests.Box Backup and SSLGeneral notesIdeally, you need to use version 0.9.7 or later of OpenSSL. If
this is installed on your system by default (and it is on most recent
releases of UNIX like OSes) then everything should just work.However, if it isn't, you have a few options.Upgrade your installationThe best option is to upgrade your installation to use 0.9.7.
Hopefully your package manager will make this easy for you. This may
require reinstallation of lots of software which depends on OpenSSL,
so may not be ideal.(But as there have been a few security flaws in OpenSSL
recently, you probably want to upgrade it anyway.)Install another OpenSSLThe second best option is to install another copy. If you
download and install from source, it will probably install into
/usr/local/ssl. You can then configure Box Backup to use it
using:./configure --with-ssl-headers=/usr/local/ssl/include --with-ssl-lib=/usr/local/ssl/libwhich will set up the various includes and libraries for
you.The configuration scripts may be a problem, depending on your
installation. See below for more information.Use the old version of OpenSSLIf you have an old version installed, the configuration script
will give you instructions on how to enable support for older
versions. Read the warnings, and please, whatever you do, don't
release binary packages or ports which enable this option.You may have issues with the configuration scripts, see
below.If you have problems with the config scriptsIf you get OpenSSL related errors with the configuration scripts,
there are two things to check:The bin directory within your OpenSSL directory is in the path
(if you have installed another version)You have an openssl.cnf file which works and can be
found.OpenSSL config fileYou need to have an openssl.cnf file. The default will generally
work well (see example at end). Make sure the openssl utility can find
it, either set the OPENSSL_CONF environment variable, or install it
into the location that is mentioned in the error messages.Example OpenSSL config file:#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#
RANDFILE = /dev/arandom
####################################################################
[ req ]
default_bits = 1024
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
attributes = req_attributes
[ req_distinguished_name ]
countryName = Country Name (2 letter code)
#countryName_default = AU
countryName_min = 2
countryName_max = 2
stateOrProvinceName = State or Province Name (full name)
#stateOrProvinceName_default = Some-State
localityName = Locality Name (eg, city)
0.organizationName = Organization Name (eg, company)
#0.organizationName_default = Internet Widgits Pty Ltd
# we can do this but it is not needed normally :-)
#1.organizationName = Second Organization Name (eg, company)
#1.organizationName_default = CryptSoft Pty Ltd
organizationalUnitName = Organizational Unit Name (eg, section)
#organizationalUnitName_default =
commonName = Common Name (eg, fully qualified host name)
commonName_max = 64
emailAddress = Email Address
emailAddress_max = 64
[ req_attributes ]
challengePassword = A challenge password
challengePassword_min = 4
challengePassword_max = 20
unstructuredName = An optional company name
[ x509v3_extensions ]
nsCaRevocationUrl = http://www.cryptsoft.com/ca-crl.pem
nsComment = "This is a comment"
# under ASN.1, the 0 bit would be encoded as 80
nsCertType = 0x40Compiling bbackupd on Windows using Visual C++This Appendix explains how to build the bbackupd daemon for Windows
using the Visual C++ compiler.If you have any problems following these instructions, please sign
up to the mailing
list and report them to us, or send an email to Chris Wilson. Thanks!Note: bbstored will not be built
with this process. bbstored is not currently supported on Windows. There
are no plans for bbstored support on Windows.ToolsYou will need quite a bit of software to make this work. All of it
is available for free on the Internet, although Visual C++ Express has
license restrictions and a time limit.Visual C++Microsoft's Visual C++ compiler and development environment are
part of their commercial product Visual Studio. Visual
Studio 2005 is supported, and 2003 should work as well.You can also download
a free copy of Visual C++ 2005 Express. This copy must be registered
(activated) within 30 days, and is free for one year.You will need the Platform
SDK to allow you to compile Windows applications.PerlDownload and install ActivePerl for
Windows, which you can probably find here.LibrariesYou will need to download and install several libraries. They
must all be built in the same directory, to be able to link
properly.Choose a directory where you will unpack and compile OpenSSL,
Zlib and Box Backup. We will call this the base directory. An example
might be:C:\Documents and Settings\Your Username\Desktop\BoxMake sure you know the full path to this directory.OpenSSLYou will need to compile OpenSSL using Visual C++. The latest
release at this time, OpenSSL 0.9.8a, does not compile with Visual
C++ 2005 out of the box, so you need a
patched version. The original
source and patch
are also available.To compile OpenSSL:Use a Windows unzipper such as WinZip (free trial) to
extract the openssl-0.9.8a-vc2005.tar.gz archive,
which you just downloaded, into the base directory.Rename the folder from openssl-0.9.8a-vc2005 to opensslOpen a command shell (run cmd.exe) and cd to the openssl directoryRun the following commands:perl Configure VC-WIN32
ms\do_ms
"c:\program files\Microsoft Visual Studio 8\Common7\Tools\vsvars32.bat"
nmake -f ms\ntdll.makZlibYou will need to download the Zlib compiled DLL.
Create a directory called zlib in
the base directory, and unzip the file you just downloaded into that
directory. You don't need to compile anything.Download Box BackupThe first version of Box Backup that's known to compile and with
Visual C++ 2005 is available on the Subversion
server. However, this version has not been extensively tested
and may be out of date.The changes are expected to be merged into the Subversion trunk
at some point, and this page should then be updated. If in doubt,
please sign up to the mailing
list and ask us.To get the source code out of Subversion you will need a Subversion
client for Windows. After installing it, open a new command
prompt, go to the base directory, and type:svn co http://bbdev.fluffy.co.uk/svn/box/chris/win32/vc2005-compile-fixes/ boxbackupThis should create a directory called boxbackup inside the base directory.Configure Box BackupOpen a command prompt, change to the base directory then
boxbackup, and run win32.bat to configure the sources. Otherwise,
Visual C++ will complain about missing files whose names start with
autogen, and missing config.h.Compile Box BackupOpen Visual C++. Choose "File/Open/Project", navigate to the
base directory, then to boxbackup\infrastructure\msvc\2005 (or
2003 if using Visual Studio 2003),
and open any project or solution file in that directory.Press F7 to compile Box Backup. If the compilation is
successful, boxbackup\Debug\bbackupd.exe will be
created.Install Box BackupCreate the destination directory, C:\Program Files\Box Backup\bbackupd.Write a configuration file, keys and certificate on a Unix
machine, and copy them into the Box
Backup directory, together with the following files from
the base directory:boxbackup\Debug\bbackupd.exeopenssl\out32dll\libeay32.dllopenssl\out32dll\ssleay32.dllzlib\zlib1.dllEnsure that the user running Box Backup can read from the
Box Backup directory, and write to
the bbackupd directory inside
it.Run Box Backup by double-clicking on it, and check that it
connects to the server. If the window opens and closes immediately,
it's probably due to a problem with the configuration file - check the
Windows Event Viewer for details.Windows ServiceBox Backup can also run as a Windows service, in which case it
will be automatically started at boot time in the background. To
install this, open a command prompt, and run:cd "C:\Program Files\Box Backup"
bbackupd.exe -iThis should output Box Backup service installed.Compilation and installation by building an RPM on
LinuxIt is very easy to build an RPM of Box Backup on Linux platforms.
This should work on all Red Hat distributions (including Fedora), SuSE,
and probably others too.Given that you have the correct development packages installed
simply execute this command (replacing the version number):rpmbuild -ta boxbackup-0.00.tgzrpmbuild will report where the packages have been written to, and
these can be installed in the normal manner.If you have never built an RPM before you should set up a convenient
build area as described in the RPM
book.If you wish to customise the package you can find the spec file in
the contrib/rpm directory.