diff options
Diffstat (limited to 'documentation/boxbackup')
| -rw-r--r-- | documentation/boxbackup/instguide.sgml | 750 |
1 files changed, 750 insertions, 0 deletions
diff --git a/documentation/boxbackup/instguide.sgml b/documentation/boxbackup/instguide.sgml new file mode 100644 index 00000000..319b08e8 --- /dev/null +++ b/documentation/boxbackup/instguide.sgml @@ -0,0 +1,750 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN" +"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"> +<book> + <title>Box Backup Installation & Configuration Guide</title> + + <preface> + <title>License</title> + + <para>Copyright (c) <YEAR>, <OWNER></para> + + <para>All rights reserved.</para> + + <para>Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are + met:</para> + + <itemizedlist> + <listitem> + <para>Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer.</para> + </listitem> + + <listitem> + <para>Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following disclaimer + in the documentation and/or other materials provided with the + distribution.</para> + </listitem> + + <listitem> + <para>Neither the name of the <ORGANIZATION> nor the names of + its contributors may be used to endorse or promote products derived + from this software without specific prior written permission</para> + </listitem> + </itemizedlist> + + <para>THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR + CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR + PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.</para> + </preface> + + <chapter> + <title>Introduction</title> + + <para>The backup daemon, bbackupd, runs on all machines to be backed up. + The store server daemon, bbstored runs on a central server. Data is sent + to the store server, which stores all data on local filesystems, that is, + only on local hard drives. Tape or other archive media is not used.</para> + + <para>The system is designed to be easy to set up and run, and cheap to + use. Once set up, there should be no need for user or administrative + intervention, apart from usual system maintenance.</para> + + <section> + <title>Client daemon</title> + + <para>bbackupd is configured with a list of directories to back up. It + has a lazy approach to backing up data. Every so often, the directories + are scanned, and new data is uploaded to the server. This new data must + be over a set age before it is uploaded. This prevents rapid revisions + of a file resulting in many uploads of the same file in a short period + of time.</para> + + <para>It can also operate in a snapshot mode, which behaves like + traditional backup software. When instructed by an external bbackupctl + program, it will upload all changed files to the server.</para> + + <para>The daemon is always running, although sleeping most of the time. + In lazy mode, it is completely self contained -- scripts running under + cron jobs are not used. The objective is to keep files backed up, not to + make snapshots of the filesystem at particular points in time + available.</para> + + <para>If an old version of the file is present on the server, a modified + version of the rsync algorithm is used to upload only the changed + portions of the file.</para> + + <para>After a new version is uploaded, the old version is still + available (subject to disc space on the server). Similarly, a deleted + file is still available. The only limit to their availability is space + allocated to this account on the server</para> + + <para>Future versions will add the ability to mark the current state of + files on the server, and restore from this mark. This will emulate the + changing of tapes in a tape backup system.</para> + + <section> + <title>Restoration</title> + + <para>Restoring files is performed using a query tool, bbackupquery. + This can be used to restore entire directories, or as an 'FTP-like' + tool to list and retrieve individual files. Old versions and deleted + files can be retrieved using this tool for as long as they are kept on + the server.</para> + </section> + + <section> + <title>Client Resource Usage</title> + + <para>bbackupd uses only a minimal amount of disc space to store + records on uploaded files -- less than 32 bytes per directory and file + over a set size threshold. However, it minimises the amount of queries + it must make to the server by storing, in memory, a data structure + which allows it to determine what data is new. It does not need to + store a record of all files, essentially just the directory names and + last modification times. This is not a huge amount of memory.</para> + + <para>If there are no changes to the directories, then the client will + not even connect to the server.</para> + </section> + </section> + + <section> + <title>Security</title> + + <para>Box Backup is designed to be secure in several ways. The data + stored on the backup store server is encrypted using secret-key + cryptography. Additionally, the transport layer is encrypted using TLS, + to ensure that the communications can't be snooped.</para> + + <section> + <title>Encryption</title> + + <para>The files, directories, filenames and file attributes are all + encrypted. By examining the stored files on the server, it is only + possible to determine the approximate sizes of a files and the tree + structure of the disc (not names, just number of files and + subdirectories in a directory). By monitoring the actions performed by + a client, it is possible to determine the frequency and approximate + scope of changes to files and directories.</para> + + <para>The connections between the server and client are encrypted + using TLS (latest version of SSL). Traffic analysis is possible to + some degree, but limited in usefulness.</para> + + <para>An attacker will not be able to recover the backed up data + without the encryption keys. Of course, you won't be able to recover + your files without the keys either, so you must make a conventional, + secure, backup of these keys.</para> + </section> + + <section> + <title>Authentication</title> + + <para>SSL certificates are used to authenticate clients. UNIX user + accounts are not used to minimise the dependence on the configuration + of the operating system hosting the server.</para> + + <para>A script is provided to run the necessary certification + authority with minimal effort.</para> + </section> + </section> + + <section> + <title>Server daemon</title> + + <para>The server daemon is designed to be simple to deploy, and run on + the cheapest hardware possible. To avoid the necessity to use expensive + hardware RAID or software RAID with complex setup, it (optionally) + stores files using RAID techniques.</para> + + <para>It does not need to run as a privileged user.</para> + + <para>Each account has a set amount of disc space allocated, with a soft + and a hard limit. If the account exceeds the soft limit, a housekeeping + process will start deleting old versions and deleted files to reduce the + space used to below the soft limit. If the backup client attempts to + upload a file which causes the store to exceed the hard limit, the + upload will be refused.</para> + </section> + </chapter> + + <chapter> + <title>Building and installing</title> + + <section> + <title>Before you start</title> + + <para>Firstly, check that all the clocks on your clients, servers and + signing machines are accurate and in sync. A disagreement in time + between a client and a server is the biggest cause of installation + difficulties, as the times in the generated certificates will cause + login failures if the start date is in the future.</para> + </section> + + <section> + <title>Box Backup compile</title> + + <para>In the following instructions, replace 0.00 with the actual + version number of the archive you have downloaded.</para> + + <para>For help building on Windows, see the <link linkend="AppB">Windows + Compile Appendix</link>. And if you want to build a Linux RPM, <link + linkend="AppC">look here</link>.</para> + + <para>You need the latest version of OpenSSL, as some of the extra APIs + it provides are required. You should have this anyway, as earlier + versions have security flaws. (If you have an earlier version installed, + the configuration script will give you instructions on enabling + experimental support for older versions.)</para> + + <para>See <link linkend="AppA">OpenSSL notes</link> for more information + on OpenSSL issues.</para> + + <para>There are some notes in the archive about compiling on various + platforms within the boxbackup-0.00 directory -- read them first. For + example, if you are compiling under Linux, look for LINUX.txt as + boxbackup-0.00/LINUX.txt after untaring the archive.</para> + + <para>Download the archive, then in that directory type</para> + + <programlisting>tar xvzf boxbackup-0.00.tgz +cd boxbackup-0.00 +./configure +make</programlisting> + + <para>The server and client will be built and packaged up for + installation on this machine, or ready to be transferred as tar files to + another machine for installation.</para> + + <para>This builds two parcels of binaries and scripts, 'backup-client' + and 'backup-server'. The generated installation scripts assumes you want + everything installed in <emphasis + role="bold">/usr/local/bin</emphasis></para> + + <para>Optionally, type <emphasis role="bold">make test</emphasis> to run + all the tests.</para> + </section> + + <section> + <title>Local installation</title> + + <para>Type <emphasis role="bold">make install-backup-client</emphasis> + to install the backup client.</para> + + <para>Type <emphasis role="bold">make install-backup-server</emphasis> + to install the backup server.</para> + </section> + + <section> + <title>Remote installation</title> + + <para>In the parcels directory, there are tar files for each parcel. The + name reflects the version and platform you have built it for.</para> + + <para>Transfer this tar file to the remote server, and unpack it, then + run the install script. For example:</para> + + <programlisting>tar xvzf boxbackup-0.00-backup-server-OpenBSD.tgz +cd boxbackup-0.00-backup-server-OpenBSD +./install-backup-server</programlisting> + </section> + + <section> + <title>Configure options</title> + + <para>You can use arguments to the configure script to adjust the + compile and link lines in the generated Makefiles, should this be + necessary for your platform. The configure script takes the usual GNU + autoconf arguments, a full list of which can be obtained with <emphasis + role="bold">--help</emphasis>. Additional options for Box Backup + include:</para> + + <informaltable> + <tgroup cols="2"> + <tbody> + <row> + <entry char="-">--enable-gnu-readline</entry> + + <entry>Use GNU readline if present. Linking Box Backup against + GNU readline may create licence implications if you then + distribute the binaries. libeditline is also supported as a safe + alternative, and is used by default if available.</entry> + </row> + + <row> + <entry>--disable-largefile</entry> + + <entry>Omit support for large files</entry> + </row> + + <row> + <entry>--with-bdb-dir=DIR</entry> + + <entry>Specify Berkeley DB location</entry> + </row> + + <row> + <entry>--with-random=FILE</entry> + + <entry>Use FILE as random number seed (normally + auto-detected)</entry> + </row> + + <row> + <entry>--with-tmp-dir=DIR</entry> + + <entry>Directory for temporary files (normally /tmp)</entry> + </row> + </tbody> + </tgroup> + </informaltable> + + <para>See <link linkend="AppA">OpenSSL notes</link> for the OpenSSL + specific options.</para> + </section> + + <section> + <title>Tests</title> + + <para>There are a number of unit tests provided. To compile and run one + type:</para> + + <programlisting>./runtest.pl bbackupd release +./runtest.pl common debug +./runtest.pl ALL</programlisting> + + <para>The runtest.pl script will compile and run the test. The first + argument is the test name, and the second the type of build. Use ALL as + a test name to run all the tests.</para> + + <para>The output from the tests is slightly muddled using this script. + If you're developing, porting or trying out new things, it might be + better to use the following scheme:</para> + + <programlisting>cd test/bbackupd +make +cd ../../debug/test/bbackupd +./t</programlisting> + + <para>or in release mode...</para> + + <programlisting>cd test/bbackupd +make -D RELEASE +cd ../../release/test/bbackupd +./t</programlisting> + + <para>(use RELEASE=1 with GNU make)</para> + + <para>I tend to use two windows, one for compilation, and one for + running tests.</para> + </section> + </chapter> + + <appendix> + <title id="AppA">Box Backup and SSL</title> + + <section> + <title>General notes</title> + + <para>Ideally, you need to use version 0.9.7 or later of OpenSSL. If + this is installed on your system by default (and it is on most recent + releases of UNIX like OSes) then everything should just work.</para> + + <para>However, if it isn't, you have a few options.</para> + + <section> + <title>Upgrade your installation</title> + + <para>The best option is to upgrade your installation to use 0.9.7. + Hopefully your package manager will make this easy for you. This may + require reinstallation of lots of software which depends on OpenSSL, + so may not be ideal.</para> + + <para>(But as there have been a few security flaws in OpenSSL + recently, you probably want to upgrade it anyway.)</para> + </section> + + <section> + <title>Install another OpenSSL</title> + + <para>The second best option is to install another copy. If you + download and install from source, it will probably install into + /usr/local/ssl. You can then configure Box Backup to use it + using:</para> + + <programlisting>./configure --with-ssl-headers=/usr/local/ssl/include --with-ssl-lib=/usr/local/ssl/lib</programlisting> + + <para>which will set up the various includes and libraries for + you.</para> + + <para>The configuration scripts may be a problem, depending on your + installation. See below for more information.</para> + </section> + + <section> + <title>Use the old version of OpenSSL</title> + + <para>If you have an old version installed, the configuration script + will give you instructions on how to enable support for older + versions. Read the warnings, and please, whatever you do, don't + release binary packages or ports which enable this option.</para> + + <para>You may have issues with the configuration scripts, see + below.</para> + </section> + </section> + + <section> + <title>If you have problems with the config scripts</title> + + <para>If you get OpenSSL related errors with the configuration scripts, + there are two things to check:</para> + + <itemizedlist> + <listitem> + <para>The bin directory within your OpenSSL directory is in the path + (if you have installed another version)</para> + </listitem> + + <listitem> + <para>You have an openssl.cnf file which works and can be + found.</para> + </listitem> + </itemizedlist> + + <section> + <title>OpenSSL config file</title> + + <para>You need to have an openssl.cnf file. The default will generally + work well (see example at end). Make sure the openssl utility can find + it, either set the OPENSSL_CONF environment variable, or install it + into the location that is mentioned in the error messages.</para> + + <para>Example OpenSSL config file:</para> + + <programlisting id="openssl.cnf" xreflabel="openssl.cnf"># +# OpenSSL example configuration file. +# This is mostly being used for generation of certificate requests. +# + +RANDFILE = /dev/arandom + +#################################################################### +[ req ] +default_bits = 1024 +default_keyfile = privkey.pem +distinguished_name = req_distinguished_name +attributes = req_attributes + +[ req_distinguished_name ] +countryName = Country Name (2 letter code) +#countryName_default = AU +countryName_min = 2 +countryName_max = 2 + +stateOrProvinceName = State or Province Name (full name) +#stateOrProvinceName_default = Some-State + +localityName = Locality Name (eg, city) + +0.organizationName = Organization Name (eg, company) +#0.organizationName_default = Internet Widgits Pty Ltd + +# we can do this but it is not needed normally :-) +#1.organizationName = Second Organization Name (eg, company) +#1.organizationName_default = CryptSoft Pty Ltd + +organizationalUnitName = Organizational Unit Name (eg, section) +#organizationalUnitName_default = + +commonName = Common Name (eg, fully qualified host name) +commonName_max = 64 + +emailAddress = Email Address +emailAddress_max = 64 + +[ req_attributes ] +challengePassword = A challenge password +challengePassword_min = 4 +challengePassword_max = 20 + +unstructuredName = An optional company name + +[ x509v3_extensions ] + +nsCaRevocationUrl = http://www.cryptsoft.com/ca-crl.pem +nsComment = "This is a comment" + +# under ASN.1, the 0 bit would be encoded as 80 +nsCertType = 0x40</programlisting> + </section> + </section> + </appendix> + + <appendix> + <title id="AppB">Compiling bbackupd on Windows using Visual C++</title> + + <para>This Appendix explains how to build the bbackupd daemon for Windows + using the Visual C++ compiler.</para> + + <para>If you have any problems following these instructions, please sign + up to the <ulink + url="http://lists.warhead.org.uk/mailman/listinfo/boxbackup">mailing + lis</ulink>t and report them to us, or send an email to <ulink + url="mailto:bbwiki@qwirx.com">Chris Wilson</ulink>. Thanks!</para> + + <para><emphasis role="bold">Note</emphasis>: bbstored will not be built + with this process. bbstored is not currently supported on Windows. There + are no plans for bbstored support on Windows.</para> + + <section> + <title>Tools</title> + + <para>You will need quite a bit of software to make this work. All of it + is available for free on the Internet, although Visual C++ Express has + license restrictions and a time limit.</para> + + <section> + <title>Visual C++</title> + + <para>Microsoft's Visual C++ compiler and development environment are + part of their commercial product <ulink + url="http://msdn.microsoft.com/vstudio/">Visual Studio</ulink>. Visual + Studio 2005 is supported, and 2003 should work as well.</para> + + <para>You can also <ulink + url="http://msdn.microsoft.com/vstudio/express/visualc/download/">download</ulink> + a free copy of Visual C++ 2005 Express. This copy must be registered + (activated) within 30 days, and is free for one year.</para> + + <para>You will need the <ulink + url="http://msdn.microsoft.com/vstudio/express/visualc/usingpsdk/">Platform + SDK</ulink> to allow you to compile Windows applications.</para> + </section> + + <section> + <title>Perl</title> + + <para>Download and install <ulink + url="http://www.activestate.com/Products/ActivePerl/">ActivePerl for + Windows</ulink>, which you can probably find <ulink + url="http://downloads.activestate.com/ActivePerl/Windows/5.6/ActivePerl-5.6.1.638-MSWin32-x86.msi">here</ulink>.</para> + </section> + + <section> + <title>Libraries</title> + + <para>You will need to download and install several libraries. They + must all be built in the same directory, to be able to link + properly.</para> + + <para>Choose a directory where you will unpack and compile OpenSSL, + Zlib and Box Backup. We will call this the base directory. An example + might be:</para> + + <programlisting>C:\Documents and Settings\Your Username\Desktop\Box</programlisting> + + <para>Make sure you know the full path to this directory.</para> + + <section> + <title>OpenSSL</title> + + <para>You will need to compile OpenSSL using Visual C++. The latest + release at this time, OpenSSL 0.9.8a, does not compile with Visual + C++ 2005 out of the box, so you need <ulink + url="http://bbdev.fluffy.co.uk/svn/box/chris/win32/support/openssl-0.9.8a-vc2005.zip">a + patched version</ulink>. The <ulink + url="http://www.openssl.org/source/openssl-0.9.8a.tar.gz">original + source</ulink> and <ulink + url="http://bbdev.fluffy.co.uk/svn/box/chris/win32/support/openssl-0.9.8a-win32fix.patch">patch</ulink> + are also available.</para> + + <para>To compile OpenSSL:</para> + + <itemizedlist> + <listitem> + <para>Use a Windows unzipper such as <ulink + url="http://www.winzip.com/">WinZip (free trial)</ulink> to + extract the <emphasis + role="bold">openssl-0.9.8a-vc2005.tar.gz</emphasis> archive, + which you just downloaded, into the base directory.</para> + </listitem> + + <listitem> + <para>Rename the folder from <emphasis + role="bold">openssl-0.9.8a-vc2005</emphasis> to <emphasis + role="bold">openssl</emphasis></para> + </listitem> + + <listitem> + <para>Open a command shell (run <emphasis + role="bold">cmd.exe</emphasis>) and <emphasis + role="bold">cd</emphasis> to the openssl directory</para> + </listitem> + + <listitem> + <para>Run the following commands:</para> + + <programlisting>perl Configure VC-WIN32 +ms\do_ms +"c:\program files\Microsoft Visual Studio 8\Common7\Tools\vsvars32.bat" +nmake -f ms\ntdll.mak</programlisting> + </listitem> + </itemizedlist> + </section> + + <section> + <title>Zlib</title> + + <para>You will need to download the <ulink + url="http://www.zlib.net/zlib123-dll.zip">Zlib compiled DLL</ulink>. + Create a directory called <emphasis role="bold">zlib</emphasis> in + the base directory, and unzip the file you just downloaded into that + directory. You don't need to compile anything.</para> + </section> + </section> + + <section> + <title>Download Box Backup</title> + + <para>The first version of Box Backup that's known to compile and with + Visual C++ 2005 is available on the <ulink + url="http://bbdev.fluffy.co.uk/svn/box/chris/win32/vc2005-compile-fixes/">Subversion + server</ulink>. However, this version has not been extensively tested + and may be out of date.</para> + + <para>The changes are expected to be merged into the <ulink + url="http://bbdev.fluffy.co.uk/svn/box/trunk">Subversion trunk</ulink> + at some point, and this page should then be updated. If in doubt, + please sign up to the <ulink + url="http://lists.warhead.org.uk/mailman/listinfo/boxbackup">mailing + list</ulink> and ask us.</para> + + <para>To get the source code out of Subversion you will need a <ulink + url="http://subversion.tigris.org/files/documents/15/25364/svn-1.2.3-setup.exe">Subversion + client for Windows</ulink>. After installing it, open a new command + prompt, go to the base directory, and type:</para> + + <programlisting>svn co http://bbdev.fluffy.co.uk/svn/box/chris/win32/vc2005-compile-fixes/ boxbackup</programlisting> + + <para>This should create a directory called <emphasis + role="bold">boxbackup</emphasis> inside the base directory.</para> + </section> + + <section> + <title>Configure Box Backup</title> + + <para>Open a command prompt, change to the base directory then + <emphasis role="bold">boxbackup</emphasis>, and run <emphasis + role="bold">win32.bat</emphasis> to configure the sources. Otherwise, + Visual C++ will complain about missing files whose names start with + <emphasis role="bold">autogen</emphasis>, and missing <emphasis + role="bold">config.h</emphasis>.</para> + </section> + + <section> + <title>Compile Box Backup</title> + + <para>Open Visual C++. Choose "File/Open/Project", navigate to the + base directory, then to <emphasis + role="bold">boxbackup\infrastructure\msvc\2005</emphasis> (or + <emphasis role="bold">2003</emphasis> if using Visual Studio 2003), + and open any project or solution file in that directory.</para> + + <para>Press F7 to compile Box Backup. If the compilation is + successful, <emphasis + role="bold">boxbackup\Debug\bbackupd.exe</emphasis> will be + created.</para> + </section> + + <section> + <title>Install Box Backup</title> + + <para>Create the destination directory, <emphasis + role="bold">C:\Program Files\Box Backup\bbackupd</emphasis>.</para> + + <para>Write a configuration file, keys and certificate on a Unix + machine, and copy them into the <emphasis role="bold">Box + Backup</emphasis> directory, together with the following files from + the base directory:</para> + + <itemizedlist> + <listitem> + <para>boxbackup\Debug\bbackupd.exe</para> + </listitem> + + <listitem> + <para>openssl\out32dll\libeay32.dll</para> + </listitem> + + <listitem> + <para>openssl\out32dll\ssleay32.dll</para> + </listitem> + + <listitem> + <para>zlib\zlib1.dll</para> + </listitem> + </itemizedlist> + + <para>Ensure that the user running Box Backup can read from the + <emphasis role="bold">Box Backup</emphasis> directory, and write to + the <emphasis role="bold">bbackupd</emphasis> directory inside + it.</para> + + <para>Run Box Backup by double-clicking on it, and check that it + connects to the server. If the window opens and closes immediately, + it's probably due to a problem with the configuration file - check the + Windows Event Viewer for details.</para> + </section> + + <section> + <title>Windows Service</title> + + <para>Box Backup can also run as a Windows service, in which case it + will be automatically started at boot time in the background. To + install this, open a command prompt, and run:</para> + + <programlisting>cd "C:\Program Files\Box Backup" +bbackupd.exe -i</programlisting> + + <para>This should output Box Backup service installed.</para> + </section> + </section> + </appendix> + + <appendix> + <title id="AppC">Compilation and installation by building an RPM on + Linux</title> + + <para>It is very easy to build an RPM of Box Backup on Linux platforms. + This should work on all Red Hat distributions (including Fedora), SuSE, + and probably others too.</para> + + <para>Given that you have the correct development packages installed + simply execute this command (replacing the version number): </para> + + <programlisting>rpmbuild -ta boxbackup-0.00.tgz</programlisting> + + <para>rpmbuild will report where the packages have been written to, and + these can be installed in the normal manner.</para> + + <para>If you have never built an RPM before you should set up a convenient + build area as described in the <ulink + url="http://www.rpm.org/max-rpm/s1-rpm-anywhere-different-build-area.html">RPM + book</ulink>.</para> + + <para>If you wish to customise the package you can find the spec file in + the contrib/rpm directory. </para> + </appendix> +</book>
\ No newline at end of file |