From e0d7e872148e574fa735d0195f4f41ce89e2af57 Mon Sep 17 00:00:00 2001 From: Per Reedtz Thomsen Date: Fri, 10 Mar 2006 19:01:23 +0000 Subject: Added rough cut of the installation guide. No DSSSL yet, but at least it can be reviewed, and edited. --- documentation/boxbackup/instguide.sgml | 750 +++++++++++++++++++++++++++++++++ 1 file changed, 750 insertions(+) create mode 100644 documentation/boxbackup/instguide.sgml (limited to 'documentation/boxbackup') diff --git a/documentation/boxbackup/instguide.sgml b/documentation/boxbackup/instguide.sgml new file mode 100644 index 00000000..319b08e8 --- /dev/null +++ b/documentation/boxbackup/instguide.sgml @@ -0,0 +1,750 @@ + + + + Box Backup Installation & Configuration Guide + + + License + + Copyright (c) <YEAR>, <OWNER> + + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are + met: + + + + Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + + + Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following disclaimer + in the documentation and/or other materials provided with the + distribution. + + + + Neither the name of the <ORGANIZATION> nor the names of + its contributors may be used to endorse or promote products derived + from this software without specific prior written permission + + + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR + CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR + PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + + + Introduction + + The backup daemon, bbackupd, runs on all machines to be backed up. + The store server daemon, bbstored runs on a central server. Data is sent + to the store server, which stores all data on local filesystems, that is, + only on local hard drives. Tape or other archive media is not used. + + The system is designed to be easy to set up and run, and cheap to + use. Once set up, there should be no need for user or administrative + intervention, apart from usual system maintenance. + +
+ Client daemon + + bbackupd is configured with a list of directories to back up. It + has a lazy approach to backing up data. Every so often, the directories + are scanned, and new data is uploaded to the server. This new data must + be over a set age before it is uploaded. This prevents rapid revisions + of a file resulting in many uploads of the same file in a short period + of time. + + It can also operate in a snapshot mode, which behaves like + traditional backup software. When instructed by an external bbackupctl + program, it will upload all changed files to the server. + + The daemon is always running, although sleeping most of the time. + In lazy mode, it is completely self contained -- scripts running under + cron jobs are not used. The objective is to keep files backed up, not to + make snapshots of the filesystem at particular points in time + available. + + If an old version of the file is present on the server, a modified + version of the rsync algorithm is used to upload only the changed + portions of the file. + + After a new version is uploaded, the old version is still + available (subject to disc space on the server). Similarly, a deleted + file is still available. The only limit to their availability is space + allocated to this account on the server + + Future versions will add the ability to mark the current state of + files on the server, and restore from this mark. This will emulate the + changing of tapes in a tape backup system. + +
+ Restoration + + Restoring files is performed using a query tool, bbackupquery. + This can be used to restore entire directories, or as an 'FTP-like' + tool to list and retrieve individual files. Old versions and deleted + files can be retrieved using this tool for as long as they are kept on + the server. +
+ +
+ Client Resource Usage + + bbackupd uses only a minimal amount of disc space to store + records on uploaded files -- less than 32 bytes per directory and file + over a set size threshold. However, it minimises the amount of queries + it must make to the server by storing, in memory, a data structure + which allows it to determine what data is new. It does not need to + store a record of all files, essentially just the directory names and + last modification times. This is not a huge amount of memory. + + If there are no changes to the directories, then the client will + not even connect to the server. +
+
+ +
+ Security + + Box Backup is designed to be secure in several ways. The data + stored on the backup store server is encrypted using secret-key + cryptography. Additionally, the transport layer is encrypted using TLS, + to ensure that the communications can't be snooped. + +
+ Encryption + + The files, directories, filenames and file attributes are all + encrypted. By examining the stored files on the server, it is only + possible to determine the approximate sizes of a files and the tree + structure of the disc (not names, just number of files and + subdirectories in a directory). By monitoring the actions performed by + a client, it is possible to determine the frequency and approximate + scope of changes to files and directories. + + The connections between the server and client are encrypted + using TLS (latest version of SSL). Traffic analysis is possible to + some degree, but limited in usefulness. + + An attacker will not be able to recover the backed up data + without the encryption keys. Of course, you won't be able to recover + your files without the keys either, so you must make a conventional, + secure, backup of these keys. +
+ +
+ Authentication + + SSL certificates are used to authenticate clients. UNIX user + accounts are not used to minimise the dependence on the configuration + of the operating system hosting the server. + + A script is provided to run the necessary certification + authority with minimal effort. +
+
+ +
+ Server daemon + + The server daemon is designed to be simple to deploy, and run on + the cheapest hardware possible. To avoid the necessity to use expensive + hardware RAID or software RAID with complex setup, it (optionally) + stores files using RAID techniques. + + It does not need to run as a privileged user. + + Each account has a set amount of disc space allocated, with a soft + and a hard limit. If the account exceeds the soft limit, a housekeeping + process will start deleting old versions and deleted files to reduce the + space used to below the soft limit. If the backup client attempts to + upload a file which causes the store to exceed the hard limit, the + upload will be refused. +
+ + + + Building and installing + +
+ Before you start + + Firstly, check that all the clocks on your clients, servers and + signing machines are accurate and in sync. A disagreement in time + between a client and a server is the biggest cause of installation + difficulties, as the times in the generated certificates will cause + login failures if the start date is in the future. +
+ +
+ Box Backup compile + + In the following instructions, replace 0.00 with the actual + version number of the archive you have downloaded. + + For help building on Windows, see the Windows + Compile Appendix. And if you want to build a Linux RPM, look here. + + You need the latest version of OpenSSL, as some of the extra APIs + it provides are required. You should have this anyway, as earlier + versions have security flaws. (If you have an earlier version installed, + the configuration script will give you instructions on enabling + experimental support for older versions.) + + See OpenSSL notes for more information + on OpenSSL issues. + + There are some notes in the archive about compiling on various + platforms within the boxbackup-0.00 directory -- read them first. For + example, if you are compiling under Linux, look for LINUX.txt as + boxbackup-0.00/LINUX.txt after untaring the archive. + + Download the archive, then in that directory type + + tar xvzf boxbackup-0.00.tgz +cd boxbackup-0.00 +./configure +make + + The server and client will be built and packaged up for + installation on this machine, or ready to be transferred as tar files to + another machine for installation. + + This builds two parcels of binaries and scripts, 'backup-client' + and 'backup-server'. The generated installation scripts assumes you want + everything installed in /usr/local/bin + + Optionally, type make test to run + all the tests. +
+ +
+ Local installation + + Type make install-backup-client + to install the backup client. + + Type make install-backup-server + to install the backup server. +
+ +
+ Remote installation + + In the parcels directory, there are tar files for each parcel. The + name reflects the version and platform you have built it for. + + Transfer this tar file to the remote server, and unpack it, then + run the install script. For example: + + tar xvzf boxbackup-0.00-backup-server-OpenBSD.tgz +cd boxbackup-0.00-backup-server-OpenBSD +./install-backup-server +
+ +
+ Configure options + + You can use arguments to the configure script to adjust the + compile and link lines in the generated Makefiles, should this be + necessary for your platform. The configure script takes the usual GNU + autoconf arguments, a full list of which can be obtained with --help. Additional options for Box Backup + include: + + + + + + --enable-gnu-readline + + Use GNU readline if present. Linking Box Backup against + GNU readline may create licence implications if you then + distribute the binaries. libeditline is also supported as a safe + alternative, and is used by default if available. + + + + --disable-largefile + + Omit support for large files + + + + --with-bdb-dir=DIR + + Specify Berkeley DB location + + + + --with-random=FILE + + Use FILE as random number seed (normally + auto-detected) + + + + --with-tmp-dir=DIR + + Directory for temporary files (normally /tmp) + + + + + + See OpenSSL notes for the OpenSSL + specific options. +
+ +
+ Tests + + There are a number of unit tests provided. To compile and run one + type: + + ./runtest.pl bbackupd release +./runtest.pl common debug +./runtest.pl ALL + + The runtest.pl script will compile and run the test. The first + argument is the test name, and the second the type of build. Use ALL as + a test name to run all the tests. + + The output from the tests is slightly muddled using this script. + If you're developing, porting or trying out new things, it might be + better to use the following scheme: + + cd test/bbackupd +make +cd ../../debug/test/bbackupd +./t + + or in release mode... + + cd test/bbackupd +make -D RELEASE +cd ../../release/test/bbackupd +./t + + (use RELEASE=1 with GNU make) + + I tend to use two windows, one for compilation, and one for + running tests. +
+ + + + Box Backup and SSL + +
+ General notes + + Ideally, you need to use version 0.9.7 or later of OpenSSL. If + this is installed on your system by default (and it is on most recent + releases of UNIX like OSes) then everything should just work. + + However, if it isn't, you have a few options. + +
+ Upgrade your installation + + The best option is to upgrade your installation to use 0.9.7. + Hopefully your package manager will make this easy for you. This may + require reinstallation of lots of software which depends on OpenSSL, + so may not be ideal. + + (But as there have been a few security flaws in OpenSSL + recently, you probably want to upgrade it anyway.) +
+ +
+ Install another OpenSSL + + The second best option is to install another copy. If you + download and install from source, it will probably install into + /usr/local/ssl. You can then configure Box Backup to use it + using: + + ./configure --with-ssl-headers=/usr/local/ssl/include --with-ssl-lib=/usr/local/ssl/lib + + which will set up the various includes and libraries for + you. + + The configuration scripts may be a problem, depending on your + installation. See below for more information. +
+ +
+ Use the old version of OpenSSL + + If you have an old version installed, the configuration script + will give you instructions on how to enable support for older + versions. Read the warnings, and please, whatever you do, don't + release binary packages or ports which enable this option. + + You may have issues with the configuration scripts, see + below. +
+
+ +
+ If you have problems with the config scripts + + If you get OpenSSL related errors with the configuration scripts, + there are two things to check: + + + + The bin directory within your OpenSSL directory is in the path + (if you have installed another version) + + + + You have an openssl.cnf file which works and can be + found. + + + +
+ OpenSSL config file + + You need to have an openssl.cnf file. The default will generally + work well (see example at end). Make sure the openssl utility can find + it, either set the OPENSSL_CONF environment variable, or install it + into the location that is mentioned in the error messages. + + Example OpenSSL config file: + + # +# OpenSSL example configuration file. +# This is mostly being used for generation of certificate requests. +# + +RANDFILE = /dev/arandom + +#################################################################### +[ req ] +default_bits = 1024 +default_keyfile = privkey.pem +distinguished_name = req_distinguished_name +attributes = req_attributes + +[ req_distinguished_name ] +countryName = Country Name (2 letter code) +#countryName_default = AU +countryName_min = 2 +countryName_max = 2 + +stateOrProvinceName = State or Province Name (full name) +#stateOrProvinceName_default = Some-State + +localityName = Locality Name (eg, city) + +0.organizationName = Organization Name (eg, company) +#0.organizationName_default = Internet Widgits Pty Ltd + +# we can do this but it is not needed normally :-) +#1.organizationName = Second Organization Name (eg, company) +#1.organizationName_default = CryptSoft Pty Ltd + +organizationalUnitName = Organizational Unit Name (eg, section) +#organizationalUnitName_default = + +commonName = Common Name (eg, fully qualified host name) +commonName_max = 64 + +emailAddress = Email Address +emailAddress_max = 64 + +[ req_attributes ] +challengePassword = A challenge password +challengePassword_min = 4 +challengePassword_max = 20 + +unstructuredName = An optional company name + +[ x509v3_extensions ] + +nsCaRevocationUrl = http://www.cryptsoft.com/ca-crl.pem +nsComment = "This is a comment" + +# under ASN.1, the 0 bit would be encoded as 80 +nsCertType = 0x40 +
+
+ + + + Compiling bbackupd on Windows using Visual C++ + + This Appendix explains how to build the bbackupd daemon for Windows + using the Visual C++ compiler. + + If you have any problems following these instructions, please sign + up to the mailing + list and report them to us, or send an email to Chris Wilson. Thanks! + + Note: bbstored will not be built + with this process. bbstored is not currently supported on Windows. There + are no plans for bbstored support on Windows. + +
+ Tools + + You will need quite a bit of software to make this work. All of it + is available for free on the Internet, although Visual C++ Express has + license restrictions and a time limit. + +
+ Visual C++ + + Microsoft's Visual C++ compiler and development environment are + part of their commercial product Visual Studio. Visual + Studio 2005 is supported, and 2003 should work as well. + + You can also download + a free copy of Visual C++ 2005 Express. This copy must be registered + (activated) within 30 days, and is free for one year. + + You will need the Platform + SDK to allow you to compile Windows applications. +
+ +
+ Perl + + Download and install ActivePerl for + Windows, which you can probably find here. +
+ +
+ Libraries + + You will need to download and install several libraries. They + must all be built in the same directory, to be able to link + properly. + + Choose a directory where you will unpack and compile OpenSSL, + Zlib and Box Backup. We will call this the base directory. An example + might be: + + C:\Documents and Settings\Your Username\Desktop\Box + + Make sure you know the full path to this directory. + +
+ OpenSSL + + You will need to compile OpenSSL using Visual C++. The latest + release at this time, OpenSSL 0.9.8a, does not compile with Visual + C++ 2005 out of the box, so you need a + patched version. The original + source and patch + are also available. + + To compile OpenSSL: + + + + Use a Windows unzipper such as WinZip (free trial) to + extract the openssl-0.9.8a-vc2005.tar.gz archive, + which you just downloaded, into the base directory. + + + + Rename the folder from openssl-0.9.8a-vc2005 to openssl + + + + Open a command shell (run cmd.exe) and cd to the openssl directory + + + + Run the following commands: + + perl Configure VC-WIN32 +ms\do_ms +"c:\program files\Microsoft Visual Studio 8\Common7\Tools\vsvars32.bat" +nmake -f ms\ntdll.mak + + +
+ +
+ Zlib + + You will need to download the Zlib compiled DLL. + Create a directory called zlib in + the base directory, and unzip the file you just downloaded into that + directory. You don't need to compile anything. +
+
+ +
+ Download Box Backup + + The first version of Box Backup that's known to compile and with + Visual C++ 2005 is available on the Subversion + server. However, this version has not been extensively tested + and may be out of date. + + The changes are expected to be merged into the Subversion trunk + at some point, and this page should then be updated. If in doubt, + please sign up to the mailing + list and ask us. + + To get the source code out of Subversion you will need a Subversion + client for Windows. After installing it, open a new command + prompt, go to the base directory, and type: + + svn co http://bbdev.fluffy.co.uk/svn/box/chris/win32/vc2005-compile-fixes/ boxbackup + + This should create a directory called boxbackup inside the base directory. +
+ +
+ Configure Box Backup + + Open a command prompt, change to the base directory then + boxbackup, and run win32.bat to configure the sources. Otherwise, + Visual C++ will complain about missing files whose names start with + autogen, and missing config.h. +
+ +
+ Compile Box Backup + + Open Visual C++. Choose "File/Open/Project", navigate to the + base directory, then to boxbackup\infrastructure\msvc\2005 (or + 2003 if using Visual Studio 2003), + and open any project or solution file in that directory. + + Press F7 to compile Box Backup. If the compilation is + successful, boxbackup\Debug\bbackupd.exe will be + created. +
+ +
+ Install Box Backup + + Create the destination directory, C:\Program Files\Box Backup\bbackupd. + + Write a configuration file, keys and certificate on a Unix + machine, and copy them into the Box + Backup directory, together with the following files from + the base directory: + + + + boxbackup\Debug\bbackupd.exe + + + + openssl\out32dll\libeay32.dll + + + + openssl\out32dll\ssleay32.dll + + + + zlib\zlib1.dll + + + + Ensure that the user running Box Backup can read from the + Box Backup directory, and write to + the bbackupd directory inside + it. + + Run Box Backup by double-clicking on it, and check that it + connects to the server. If the window opens and closes immediately, + it's probably due to a problem with the configuration file - check the + Windows Event Viewer for details. +
+ +
+ Windows Service + + Box Backup can also run as a Windows service, in which case it + will be automatically started at boot time in the background. To + install this, open a command prompt, and run: + + cd "C:\Program Files\Box Backup" +bbackupd.exe -i + + This should output Box Backup service installed. +
+
+ + + + Compilation and installation by building an RPM on + Linux + + It is very easy to build an RPM of Box Backup on Linux platforms. + This should work on all Red Hat distributions (including Fedora), SuSE, + and probably others too. + + Given that you have the correct development packages installed + simply execute this command (replacing the version number): + + rpmbuild -ta boxbackup-0.00.tgz + + rpmbuild will report where the packages have been written to, and + these can be installed in the normal manner. + + If you have never built an RPM before you should set up a convenient + build area as described in the RPM + book. + + If you wish to customise the package you can find the spec file in + the contrib/rpm directory. + + \ No newline at end of file -- cgit v1.2.3