path: root/documentation/boxbackup/adminguide.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
<!ENTITY __ExceptionCodes__elfjz3fu SYSTEM "ExceptionCodes.xml">
]>
<book>
  <title>Box Backup administrator's guide</title>

  <preface>
    <title>License</title>

    <para>Copyright © 2003 - 2006, Ben Summers and contributors. All rights
    reserved.</para>

    <para>Redistribution and use in source and binary forms, with or without
    modification, are permitted provided that the following conditions are
    met:</para>

    <itemizedlist>
      <listitem>
        <para>Redistributions of source code must retain the above copyright
        notice, this list of conditions and the following disclaimer.</para>
      </listitem>

      <listitem>
        <para>Redistributions in binary form must reproduce the above
        copyright notice, this list of conditions and the following disclaimer
        in the documentation and/or other materials provided with the
        distribution.</para>
      </listitem>

      <listitem>
        <para>Neither the name of the authors nor the names of its
        contributors may be used to endorse or promote products derived from
        this software without specific prior written permission</para>
      </listitem>
    </itemizedlist>

    <para>THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
    TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
    PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
    CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
    EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
    PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
    PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
    LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
    NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
    SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.</para>
  </preface>

  <chapter>
    <title>Configuration</title>

    <section>
      <title>System configuration</title>

      <section>
        <title>Server</title>

        <para>After you've downloaded and compiled the programs you need to
        install the programs on your server. As root do the following:</para>

        <programlisting>make install-backup-server</programlisting>

        <para>This assumes that you are installing on the same server that you
        compiled the software on. If not, copy the
        boxbackup-x.xx-backup-server-OSNAME.tgz file to the server you want to
        run on, and install there. For example (on Mac OS X):</para>

        <programlisting>tar zxvf boxbackup-0.10-server-darwin8.5.0.tgz
cd boxbackup-0.10-server-darwin8.5.0
./install-backup-server</programlisting>

        <para>Then create the user for the backup daemon on the server:</para>

        <programlisting>useradd _bbstored</programlisting>

        <para>BoxBackup supports Raid for the backup store. There are some
        configuration options. Use the following command if you want to create
        a simple server WITHOUT Raid protection:</para>

        <programlisting>mkdir /tmp/boxbackupRepository                        # Create the directory
chown _bbstored /tmp/boxbackupRepository/             # Change the owner to the new boxbackup daemon user

/usr/local/bin/raidfile-config /etc/box/  1024 /tmp/boxbackupRepository

#substitute 1024 with the desired blocksize
#substitute /tmp/boxbackupRepository with a directory that exists where you want the backup store located
#/usr/local/bin/raidfile-config --help shows you the options</programlisting>

        <para>Then create the configuration file /etc/box/bbstored.conf The
        hostname is tricky as it is used for two things: The name of the
        server in the certificate and the address the server is listening on.
        Since you might be using NAT, might move the server around or the
        domain name might change, choose a name that describes the server.
        When the network address of the server changes, you need to update the
        ListenAddresses directive in the /etc/box/bbstored.conf file.</para>

        <programlisting>/usr/local/bin/bbstored-config /etc/box hostname _bbstored</programlisting>

        <para>This last step outputs 5 instructions that you must execute to
        the letter. A lot of questions are raised on the mailing list because
        these steps have not been followed properly.</para>

        <para>TODO: Expand on this. Explain the 5 steps in detail.</para>

        <para>If you want to run the system as a non-root user, look <link
        linkend="WORoot">here</link>.</para>
      </section>

      <section>
        <title>Certificate Management</title>

        <para>There are two steps involved to create an account. You need to
        create the account on the server, and sign a certificate to give the
        client permission to connect to the server.</para>

        <para>Running a Certification Authority for TLS (SSL) connections is
        not trivial. However, a script to does most of the work in a way which
        should be good enough for most deployments.</para>

        <para><emphasis role="bold">Important Note:</emphasis> The certificate
        authority directory is intended to be stored on another server. It
        should not be kept on the backup server to limit the impact of a
        server compromise. The instructions and the script assume that it will
        be kept elsewhere, so will ask you to copy files to and from the
        CA.</para>

        <para><emphasis role="bold">Clock time warning</emphasis>: SSL
        certificates contain validity dates, including a "valid from" time. If
        the clock on the machine which signs the certificates is not
        syncronised to the clocks of the machines using these certificates,
        you will probably get strange errors until the start time is reached
        on all machines. If you get strange errors when attempting to use new
        certificates, check the clocks! You will probably just need to wait a
        while until the certificates become valid, rather than having to
        regenerate them.</para>

        <section>
          <title>Set up a CA</title>

          <para>It's best to do this on a machine other than your server,
          probably without direct network access. The contents of this
          directory control who can access your backup store server.</para>

          <para>To setup the basic key structure, do the following:</para>

          <programlisting>/usr/local/bin/bbstored-certs ca init</programlisting>

          <para>(See <link linkend="instguide.xml">OpenSSL notes</link> if you
          get an OpenSSL error)</para>

          <para>This creates the directory 'ca' in the current directory, and
          initialises it with basic keys.</para>
        </section>

        <section>
          <title>Sign a server certificate</title>

          <para>When you use the bbstored-config script to set up a config
          file for a server, it will generate a certificate request (CSR) for
          you. Transfer it to the machine with your CA, then do:</para>

          <programlisting>/usr/local/bin/bbstored-certs ca sign-server hostname-csr.pem</programlisting>

          <para>This signs the certificate for the server. Follow the
          instructions in the output on which files to install on the server.
          The CSR file is now no longer needed. Make sure you run this command
          from the directory above the directory 'ca'.</para>

          <para>TODO: Explain instructions in output.</para>
        </section>

        <section>
          <title>Set up an account</title>

          <para>Choose an account number for the user. This must be unique on
          the server, and is presented as a 31 bit number in hex greater than
          0, for example, 1 or 75AB23C. Then on the backup store server,
          create the account with:</para>

          <programlisting>/usr/local/bin/bbstoreaccounts create 75AB23C 0 4096M 4505M</programlisting>

          <para>This looks complicated. The numbers are, in order:</para>

          <itemizedlist>
            <listitem>
              <para>The account number allocated (hex)</para>
            </listitem>

            <listitem>
              <para>The RAID disc set (0 if you use raidfile-config and don't
              add a new set)</para>
            </listitem>

            <listitem>
              <para>Soft limit (size)</para>
            </listitem>

            <listitem>
              <para>Hard limit (size)</para>
            </listitem>
          </itemizedlist>

          <para>The sizes are are specified in Mb, Gb, or blocks, depending on
          the suffix. 1M specifies 1 Mb, 1G specifies 1 Gb, and 1B specifies 1
          block, the size of which depends on how you have configured the
          raidfile system with raidfile-config.</para>

          <para>In this example, I have allocated 4Gb (assuming you use 2048
          byte blocks as per my example) as the soft limit, and 4Gb + 10% as
          the hard limit.</para>

          <para>NOTE The sizes specified here are pre-RAID. So if you are
          using userland RAID, you are actually allocating two-thirds of this
          amount. This means that, when you take compression into account,
          that if you allocate 2Gb on the server, it'll probably hold about
          2Gb of backed up files (depending on the compressability of those
          files).</para>

          <para>The backup client will (voluntarily) try not to upload more
          data than is allowed by the soft limit. The store server will refuse
          to accept a file if it would take it over the hard limit, and when
          doing housekeeping for this account, try and delete old versions and
          deleted files to reduce the space taken to below the soft
          limit.</para>

          <para>This command will create some files on disc in the raid file
          directories (if you run as root, the utility will change to the user
          specified in the bbstored.conf file to write them) and update the
          accounts file. A server restart is not required.</para>

          <para>NOTE If you get a message saying 'Exception: RaidFile (2/8)',
          the directories you specified in the raidfile.conf are not writable
          by the _bbstored user -- fix it, and try again.</para>

          <para>Finally, tell the user their account number, and the hostname
          of your server. They will use this to set up the backup client, and
          send you a CSR. This has the account number embedded in it, and you
          should be sure that it has the right account number in it.</para>

          <para>Sign this CSR with this command:</para>

          <programlisting>/usr/local/bin/bbstored-certs ca sign 75AB23C-csr.pem</programlisting>

          <para>Don't forget to check that the embedded account number is
          correct! Then send the two files back to the user, as instructed by
          the script.</para>

          <para>Please read the Troubleshooting page if you have
          problems.</para>

          <para>TODO: Link to troubleshooting...</para>
        </section>
      </section>

      <section>
        <title>Log Files</title>

        <para>You may wish to see what's going on with the server. Edit
        /etc/syslog.conf, and add:</para>

        <programlisting>local6.info                         /var/log/box
local5.info                         /var/log/raidfile</programlisting>

        <para><emphasis role="bold">Note:</emphasis> Separators must be tabs,
        otherwise these entries will be ignored.</para>

        <programlisting>touch /var/log/box
touch /var/log/raidfile</programlisting>

        <para>Set up log rotation, by adding in /etc/newsyslog.conf:</para>

        <programlisting>/var/log/box                644  7    2000 *     Z
/var/log/raidfile           644  7    2000 *     Z</programlisting>

        <para>Then restart syslogd.</para>
      </section>

      <section>
        <title>Configuring a client</title>

        <para>Before you can do any configuration, you need to know the
        hostname of the server you will be using, and your account number on
        that server.</para>

        <para>Later in the process, you will need to send a certificate
        request to the administrator of that server for it to be
        signed.</para>

        <para>Installation is covered in the compiling and installing section.
        You only need the backup-client parcel.</para>

        <para>It is important that you read all the output of the config
        scripts. See the end of this page for an example.</para>

        <para>The backup client has to be run as root, because it needs to
        read all your files to back them up, although it is possible to back
        up a single user's files by running it as that user. (Tip: specify a
        directory other than /etc/box, and then give the alternate config file
        as the first argument to bbackupd). However, it will fall over if you
        don't give yourself read access to one of your files.</para>

        <section>
          <title id="BasicConfig">Basic configuration</title>

          <para>Run the bbackupd-config script to generate the configuration
          files and generate a private key and certificate request.</para>

          <programlisting>/usr/local/bin/bbackupd-config /etc/box lazy <emphasis
              role="bold">999 hostname</emphasis> /var/bbackupd <emphasis
              role="bold">/home</emphasis></programlisting>

          <para>(See <link linkend="instguide.sgml">OpenSSL notes</link> if
          you get an OpenSSL error)</para>

          <para>The items in bold need to be changed. In order, they are the
          account number, the hostname of the server you're using, and
          finally, the directories you want backed up. You can include as many
          you want here.</para>

          <para>However, the directories you specify must not contain other
          mounted file systems within them at any depth. Specify them
          separately, one per mount point. No checks are currently made to
          catch bad configuration of this nature!</para>

          <para>You may also want to consider changing the mode from lazy to
          snapshot, depending on what your system is used for.</para>

          <itemizedlist>
            <listitem>
              <para><emphasis role="bold">lazy</emphasis>. This mode regularly
              scans the files, with only a rough schedule. It uploads files as
              and when they are changed, if the latest version is more than a
              set age. This is good for backing up user's documents stored on
              a server, and spreads the load out over the day.</para>
            </listitem>

            <listitem>
              <para><emphasis role="bold">snapshot</emphasis>. This mode
              emulates the traditional backup behaviour of taking a snapshot
              of the filesystem. The backup daemon does absolutely nothing
              until it is instructed to make a backup using the bbackupctl
              utility (probably as a cron job), at which point it uploads all
              files which have been changed since the last time it
              uploaded.</para>
            </listitem>
          </itemizedlist>

          <para>When you run the config script, it will tell you what you need
          to do next. Don't forget to read all the output. An example is shown
          at the end of this page, but the instructions for your installation
          may be different.</para>
        </section>

        <section>
          <title>Certificates</title>

          <para>After you have sent your certificate request off to the server
          administrator and received your certificate and CA root back,
          install them where instructed by the bbackupd-config script during
          basic bbackupd configuration.</para>

          <para>You can then run the daemon (as root) by typing
          /usr/local/bin/bbackupd, and of course, adding it to your system's
          startup scripts. The first time it's run it will upload everything.
          Interrupting it and restarting it will only upload files which were
          not uploaded before - it's very tolerant.</para>

          <para>If you run in snapshot mode, you will need to add a cron job
          to schedule backups. The config script will tell you the exact
          command to use for your system.</para>

          <para>Please read the Troubleshooting page if you have
          problems.</para>

          <para>Remember to make a traditional backup of the keys file, as
          instructed. You cannot restore files without it.</para>

          <para>It is recommended that you backup up all of /etc/box as it
          will make things easier if you need to restore files. But only the
          keys are absolutely essential.</para>

          <para>If you want to see what it's doing in more detail (probably a
          good idea), follow the instructions in the server setup to create
          new log files with syslog. Adding and removing backed up
          locations.</para>

          <para>By editing the /etc/box/bbackupd.conf file, you can add and
          remove directories to back up -- see comments in this file for help.
          Send bbackupd a HUP signal after you modify it.</para>

          <para>When you remove a location, it will not be marked as deleted
          immediately. Instead, bbackupd waits about two days before doing so,
          just in case you change your mind. After this, it will be eventually
          removed from the store by the housekeeping process. Run as
          root.</para>

          <para>The backup client is designed to be run as root. It is
          possible to run without root, but this is not recommended. Clock
          synchronisation for file servers.</para>

          <para>If you are using the backup client to backup a filesystem
          served from a fileserver, you should ideally ensure that the
          fileserver clocks are synchronised with the fileserver.</para>

          <para>bbackupd will cope perfectly well if the clocks are not
          synchronised. Errors up to about half an hour cause no problems.
          Larger discrepancies cause a loss of efficiency and the potential to
          back up a file during a write process.</para>

          <para>There is a configuration parameter MaxFileTimeInFuture, which
          specifies how far in the future a file must be for it to be uploaded
          as soon as it is seen. You should not need to adjust this (default
          is 2 days). Instead, get those clocks synchronised. Excluding files
          and directories from the backup.</para>

          <para>Within the bbackupd.conf file, there is a section named
          BackupLocations which specifies which locations on disc should be
          backed up. It has subsections, each of which is in the
          format:</para>

          <programlisting> name
 {
    Path = /path/of/directory
    (optional exclude directives)
 }</programlisting>

          <para><emphasis role="bold">name</emphasis> is derived from the Path
          by the config script, but should merely be unique.</para>

          <para>The exclude directives are of the form:</para>

          <programlisting>[Exclude|AlwaysInclude][File|Dir][|sRegex] = regex or full pathname</programlisting>

          <para>(The regex suffix is shown as 'sRegex' to make File or Dir
          plural)</para>

          <para>For example:</para>

          <programlisting> ExcludeDir = /home/guest-user
 ExcludeFilesRegex = *.(mp3|MP3)\$
 AlwaysIncludeFile = /home/username/veryimportant.mp3</programlisting>

          <para>This excludes the directory /home/guest-user from the backup
          along with all mp3 files, except one MP3 file in particular.</para>

          <para>In general, Exclude excludes a file or directory, unless the
          directory is explicitly mentioned in a AlwaysInclude
          directive.</para>

          <para>If a directive ends in Regex, then it is a regular expression
          rather than a explicit full pathname. See</para>

          <programlisting> man 7 re_format</programlisting>

          <para>for the regex syntax on your platform.</para>
        </section>

        <section>
          <title>Example configuration output</title>

          <para>This is an example of output from the bbstored-config
          script.</para>

          <para><emphasis role="bold">Important</emphasis>: Follow the
          instructions output by your script, not the ones here -- they may be
          different for your system.</para>

          <programlisting>/usr/local/bin/bbackupd-config /etc/box lazy 51 server.example.com /var/bbackupd /home /etc/samba

Setup bbackupd config utility.

Configuration:
   Writing configuration file: /etc/box/bbackupd.conf
   Account: 51
   Server hostname: server.example.com
   Directories to back up:
      /home
      /etc/samba

Note: If other file systems are mounted inside these directories, then problems may occur
with files on the store server being renamed incorrectly. This will cause efficiency
problems, but not affect the integrity of the backups.

WARNING: Directories not checked against mountpoints. Check mounted filesystems manually.

Creating /etc/box...
Creating /etc/box/bbackupd
Generating private key...
 [OpenSSL output omitted]

Generating keys for file backup
Writing notify script /etc/box/bbackupd/NotifyStoreFull.sh
Writing configuration file /etc/box/bbackupd.conf

===================================================================

bbackupd basic configuration complete.

What you need to do now...

1) Make a backup of /etc/box/bbackupd/51-FileEncKeys.raw
   This should be a secure offsite backup.
   Without it, you cannot restore backups. Everything else can
   be replaced. But this cannot.
   KEEP IT IN A SAFE PLACE, OTHERWISE YOUR BACKUPS ARE USELESS.

2) Send /etc/box/bbackupd/51-csr.pem
   to the administrator of the backup server, and ask for it to
   be signed.

3) The administrator will send you two files. Install them as
      /etc/box/bbackupd/51-cert.pem
      /etc/box/bbackupd/serverCA.pem
   after checking their authenticity.

4) You may wish to read the configuration file
      /etc/box/bbackupd.conf
   and adjust as appropraite.
   
   There are some notes in it on excluding files you do not
   wish to be backed up.

5) Review the script
      /etc/box/bbackupd/NotifyStoreFull.sh
   and check that it will email the right person when the store
   becomes full. This is important -- when the store is full, no
   more files will be backed up. You want to know about this.

6) Start the backup daemon with the command
      /usr/local/bin/bbackupd
   in /etc/rc.local, or your local equivalent.
   Note that bbackupd must run as root.

===================================================================</programlisting>

          <para>Remember to make a secure, offsite backup of your backup keys,
          as described in <link linkend="BasicConfig">Basic
          configuration</link> above. If you do not,, and that key is lost,
          you have no backups.</para>
        </section>
      </section>
    </section>
  </chapter>

  <chapter>
    <title>Administration</title>

    <para>This chapter deals with the dauily running and management of the Box
    Backup system. It explains most day-to-day tasks.</para>

    <section>
      <title>Regular Maintenance</title>

      <para>The steps involved in maintaining and keeping the backup sets
      healthy are outlined in this section.</para>

      <section>
        <title>Controlling a backup client</title>

        <para>The bbackupctl program sends control commands to the bbackupd
        daemon. It must be run as the same user as the daemon, and there is no
        exception for root.</para>

        <para>The command line syntax is:</para>

        <programlisting>/usr/local/bin/bbackupctl [-q] [-c config-file] command</programlisting>

        <para>The -q option reduces the amount of output the program emits,
        and -c allows an alternative configuration file to be
        specified.</para>

        <para>Valid commands are:</para>

        <itemizedlist>
          <listitem>
            <para><emphasis role="bold">terminate</emphasis></para>

            <para>Stop the bbackupd daemon now (equivalent to kill)</para>
          </listitem>

          <listitem>
            <para><emphasis role="bold">reload</emphasis></para>

            <para>Reload the configuration file (equivalent to kill
            -HUP)</para>
          </listitem>

          <listitem>
            <para><emphasis role="bold">sync</emphasis></para>

            <para>Connect to the server and synchronise files now</para>
          </listitem>
        </itemizedlist>

        <para><emphasis role="bold">bbackupctl</emphasis> communicates with
        the server via a UNIX domain socket, specified in bbackupd.conf with
        the CommandSocket directive. This does not need to be specified, and
        <emphasis role="bold">bbackupd</emphasis> will run without the command
        socket, but in this case bbackupctl will not be able to communicate
        with the daemon.</para>

        <para>Some platforms cannot check the user id of the connecting
        process, so this command socket becomes a denial of service security
        risk. <emphasis role="bold">bbackupd</emphasis> will warn you when it
        starts up if this is the case on your platform, and you should
        consider removing the CommandSocket directive on these
        platforms.</para>
      </section>

      <section>
        <title>Using bbackupctl to perform snapshots</title>

        <para><emphasis role="bold">bbackupctl</emphasis>'s main purpose is to
        implement snapshot based backups, emulating the behaviour of
        traditional backup software.</para>

        <para>Use bbackupd-config to write a configuration file in snapshot
        mode, and then run the following command as a cron job.</para>

        <programlisting>/usr/local/bin/bbackupctl -q sync</programlisting>

        <para>This will cause the backup daemon to upload all changed files
        immediately. <emphasis role="bold">bbackupctl</emphasis> will exit
        almost immediately, and will not output anything unless there is an
        error.</para>
      </section>

      <section>
        <title>Checking storage space used on the server</title>

        <section>
          <title>From the client machine</title>

          <para>bbackupquery can tell you how much space is used on the server
          for this account. Either use the usage command in interactive mode,
          or type:</para>

          <programlisting>/usr/local/bin/bbackupquery -q usage quit</programlisting>

          <para>to show the space used as a single command.</para>
        </section>

        <section>
          <title>On the server</title>

          <para>bbstoreaccounts allows you to query the space used, and change
          the limits. To display the space used on the server for an account,
          use:</para>

          <programlisting>/usr/local/bin/bbstoreaccounts info 75AB23C</programlisting>

          <para>To adjust the soft and hard limits on an account, use:</para>

          <programlisting>/usr/local/bin/bbstoreaccounts setlimit 75AB23C new-soft-limit new-hard-limit</programlisting>

          <para>You do not need to restart the server.</para>
        </section>
      </section>

      <section>
        <title>Verify and restore files</title>

        <para>Backups are no use unless you can restore them. The bbackupquery
        utility does this and more.</para>

        <para>You don't provide any login information to it, as it just picks
        up the data it needs from /etc/box/bbackupd.conf. You should run it as
        root so it can find everything it needs.</para>

        <para>Full documentation can be found in the <link
        linkend="bbackupquery.xml">bbackupquery man page</link>. It follows
        the model of a command line sftp client quite closely.</para>

        <para>TODO: Link to bbackupquery man-page here.</para>

        <para>On systems where GNU readline is available (by default) it uses
        that for command line history and editing. Otherwise it falls back to
        very basic UNIX text entry.</para>

        <para>TODO: Did the readline dependency change to editline?</para>

        <section>
          <title>Using bbackupquery</title>

          <para>bbackupquery is the tool you use to verify, restore and
          investigate your backup files with. When invoked, it simply logs
          into the server using the certificates you have listed in
          bbackupd.conf.</para>

          <para>After you run bbackupquery, you will see a prompt, allowing
          you to execute commands. The list (or ls) command lets you view
          files in the store. It works much like unix ls, but with different
          options. An example:</para>

          <programlisting>[pthomsen@host bbackupquery]$ bbackupquery 
Box Backup Query Tool  v0.10, (c) Ben Summers and contributors 2003-2006
Using configuration file /etc/box/bbackupd.conf
Connecting to store...
Handshake with store...
Login to store...
Login complete.

Type "help" for a list of commands.

query &gt; ls
00000002 -d---- mp3
00000003 -d---- video
00000004 -d---- home-pthomsen
00000005 -d---- root
query &gt;   </programlisting>

          <para>The ls commands shows the directories that are backed up. Now
          we'll take a closer look at the home-pthomsen directory:</para>

          <programlisting>query &gt; cd home-pthomsen
query &gt; ls
00002809 f----- sample.tiff
0000280a f----- s3.tiff
0000280b f----- s4.tiff
0000280d f----- s2.tiff
0000280e f----- foo.pdf
0000286c f----- core.28720
0000339a -d---- .emacs.d
0000339d -d---- bbackup-contrib
00003437 f----- calnut.compare.txt
0000345d f----- DSCN1783.jpg
0000345e f----- DSCN1782.jpg
query &gt;</programlisting>

          <para>The ls command takes the following options;</para>

          <itemizedlist>
            <listitem>
              <para><emphasis role="bold">-r </emphasis>-- recursively list
              all files</para>
            </listitem>

            <listitem>
              <para><emphasis role="bold">-d</emphasis> -- list deleted
              files/directories</para>
            </listitem>

            <listitem>
              <para><emphasis role="bold">-o</emphasis> -- list old versions
              of files/directories</para>
            </listitem>

            <listitem>
              <para><emphasis role="bold">-I</emphasis> -- don't display
              object ID</para>
            </listitem>

            <listitem>
              <para><emphasis role="bold">-F </emphasis>-- don't display
              flags</para>
            </listitem>

            <listitem>
              <para><emphasis role="bold">-t </emphasis>-- show file
              modification time (and attr mod time if has the object has
              attributes, ~ separated)</para>
            </listitem>

            <listitem>
              <para><emphasis role="bold">-s</emphasis> -- show file size in
              blocks used on server (only very approximate indication of size
              locally)</para>
            </listitem>
          </itemizedlist>

          <para>The flags displayed from the ls command are as follows:</para>

          <simplelist>
            <member>f = file</member>

            <member>d = directory</member>

            <member>X = deleted</member>

            <member>o = old version</member>

            <member>R = remove from server as soon as marked deleted or
            old</member>

            <member>a = has attributes stored in directory record which
            override attributes in backup file</member>
          </simplelist>
        </section>

        <section>
          <title>Verify backups</title>

          <para>Since this system is not yet 100% production-ready, you'll be
          keen to verify that your backups are correct. This is easy:</para>

          <programlisting>/usr/local/bin/bbackupquery "compare -a" quit</programlisting>

          <para>It will report all the differences between the store and the
          files on disc. It will download everything, so may take a while. You
          should expect to see some differences on a typical compare, because
          files which have recently changed are unlikely to have been uploaded
          yet. Consider checking the timestamps on the files, or keeping a
          record of these messages and comparing them with a future
          verification.</para>

          <para>If you would like to do a "quick" check which just downloads
          file checksums and compares against that, then do:</para>

          <programlisting>/usr/local/bin/bbackupquery "compare -aq" quit</programlisting>

          <para>However, this does not check that the file attributes are
          correct, and since the checksums are generated on the client they
          may not reflect the data on the server if there is a problem -- the
          server cannot check the encrypted contents. View this as a good
          indication, rather than a definite check that your backup verifies
          correctly.</para>

          <para>You may wish to run either one as a cron job while testing
          this system.</para>
        </section>

        <section>
          <title>Restore backups</title>

          <para>You will need the keys file created when you configured the
          server. Without it, you cannot restore the files; this is the
          downside of encrypted backups. However, by keeping the small keys
          file safe, you indirectly keep your entire backup safe.</para>

          <para>The first step is to recreate the configuration of the backup
          client. It's probably best to have stored the /etc/box directory
          with your keys. But if you're recreating it, all you really need is
          to have got the login infomation correct (ie the certs and
          keys).</para>

          <para>Don't run bbackupd yet! It will mark all your files as deleted
          if you do, which is not hugely bad in terms of losing data, just a
          major inconvenience. (This assumes that you are working from a blank
          slate. If you want to restore some files to a different location,
          it's fine to restore while bbackupd is running, just do it outside a
          backed up directory to make sure it doesn't start uploading the
          restored files.)</para>

          <para>Type:</para>

          <programlisting>/usr/local/bin/bbackupquery</programlisting>

          <para>to run it in interactive mode.</para>

          <para>Type:</para>

          <programlisting>list</programlisting>

          <para>to see a list of the locations stored on the server.</para>

          <para>For each location you want to restore, type:</para>

          <programlisting>restore name-on-server local-dir-name</programlisting>

          <para>The directory specified by local-dir-name must not exist yet.
          If the restore is interrupted for any reason, repeat the above
          steps, but add the <emphasis role="bold">-r</emphasis> flag to the
          restore command to tell it to resume.</para>
        </section>

        <section>
          <title>Retrieving deleted and old files</title>

          <para>Box Backup makes old versions of files and files you have
          deleted available, subject to there being enough disc space on the
          server to hold them.</para>

          <para>This is how to retrieve them using bbackupquery. Future
          versions will make this far more user-friendly.</para>

          <para>Firstly, run bbackupquery in interactive mode. It behaves in a
          similar manner to a command line sftp client.</para>

          <programlisting>/usr/local/bin/bbackupquery</programlisting>

          <para>Then navigate to the directory containing the file you want,
          using list, cd and pwd.</para>

          <programlisting>query &gt; cd home/profiles/USERNAME</programlisting>

          <para>List the directory, using the "o" option to list the files
          available without filtering out everything apart from the current
          version. (if you want to see deleted files as well, use list
          -odt)</para>

          <programlisting>query &gt; list -ot
00000078 f--o- 2004-01-21T20:17:48 NTUSER.DAT
00000079 f--o- 2004-01-21T20:17:48 ntuser.dat.LOG
0000007a f--o- 2004-01-21T17:55:12 ntuser.ini
0000007b f---- 2004-01-12T15:32:00 ntuser.pol
0000007c -d--- 1970-01-01T00:00:00 Templates
00000089 -d--- 1970-01-01T00:00:00 Start Menu
000000a0 -d--- 1970-01-01T00:00:00 SendTo
000000a6 -d--- 1970-01-01T00:00:00 Recent
00000151 -d--- 1970-01-01T00:00:00 PrintHood
00000152 -d--- 1970-01-01T00:00:00 NetHood
00000156 -d--- 1970-01-01T00:00:00 My Documents
0000018d -d--- 1970-01-01T00:00:00 Favorites
00000215 -d--- 1970-01-01T00:00:00 Desktop
00000219 -d--- 1970-01-01T00:00:00 Cookies
0000048b -d--- 1970-01-01T00:00:00 Application Data
000005da -d--- 1970-01-01T00:00:00 UserData
0000437e f--o- 2004-01-24T02:45:43 NTUSER.DAT
0000437f f--o- 2004-01-24T02:45:43 ntuser.dat.LOG
00004380 f--o- 2004-01-23T17:01:29 ntuser.ini
00004446 f--o- 2004-01-24T02:45:43 NTUSER.DAT
00004447 f--o- 2004-01-24T02:45:43 ntuser.dat.LOG
000045f4 f---- 2004-01-26T15:54:16 NTUSER.DAT
000045f5 f---- 2004-01-26T15:54:16 ntuser.dat.LOG
000045f6 f---- 2004-01-26T16:54:31 ntuser.ini</programlisting>

          <para>(this is a listing from a server which is used as a Samba
          server for a network of Windows clients.) You now need to fetch the
          file using it's ID, rather than it's name. The ID is the hex number
          in the first column. Fetch it like this:</para>

          <programlisting>query &gt; get -i 0000437e NTUSER.DAT
Object ID 0000437e fetched successfully.</programlisting>

          <para>The object is now available on your local machine. You can use
          lcd to move around, and sh ls to list directories on your local
          machine.</para>
        </section>
      </section>
    </section>

    <section>
      <title id="FixCorruptions">Fixing corruptions of store data</title>

      <para>This section gives help on what to do if your server has suffered
      corruption, for example, after an unclean shutdown or other OS or
      hardware problem.</para>

      <para>In general, as updates to the store are made in an atomic manner,
      the most likely result is wasted disc space. However, if really bad
      things happen, or you believe that there is a lot of wasted space, then
      these instructions will help to restore your data.</para>

      <para>You know you will need to do something if you get strange errors,
      and bbackupd attempts to contact the server every 100 seconds or so. Or
      if one of the discs in your RAID disc set has failed.</para>

      <para>After following these instructions, the end result will be that
      bbackupquery will be able to see all the files which were stored on your
      server, and retrieve them. Some of them may be in lost+found directories
      in the root of the store (or in their original position if they have
      been moved) but they will all be able to be retrieved.</para>

      <para>After you have retrieved the files you want, bbackupd will upload
      new versions where necessary, and after about two days, mark any
      lost+found directories as deleted. Finally, those directories will be
      removed by the housekeeping process on the server.</para>

      <para>These instructions assume you're working on account 1234,
      subsitute this for whatever account you're actually working on. These
      will need to be repeated for all affected accounts.</para>

      <section>
        <title>Stop bbackupd</title>

        <para>First, make sure that bbackupd is not running on the client
        machine for the account you are going to recover. Use kill to
        terminate it. This step is not strictly necessary, but is recommended.
        During any checks on the account, bbackupd will be unable to log in,
        and after they are complete, the account is marked as changed on the
        server so bbackupd will perform a complete scan.</para>
      </section>

      <section>
        <title>Are you using RAID on the server?</title>

        <para>At the moment, the raidfile recovery tools have not been
        written. However, when two out of three files are available, the
        server will run succesfully, even if it complains a lot in the logs.
        So, your best bet here is to fix the accounts, if necessary, and
        retrieve any files you need. Then move the old store directories aside
        (in case you need them) and start afresh with new accounts, and let
        the clients upload all their data again. These utilities will be
        written shortly!</para>

        <para>TODO: Is this true anymore???</para>
      </section>

      <section>
        <title>Check and fix the account</title>

        <para>First, run the check utility, and see what errors it
        reports.</para>

        <programlisting>/usr/local/bin/bbstoreaccounts check 1234</programlisting>

        <para>This will take some time, and use a fair bit of memory (about 16
        bytes per file and directory). If the output looks plausible and
        reports errors which need fixing, run it again but with the fix
        flag:</para>

        <programlisting>/usr/local/bin/bbstoreaccounts check 1234 fix</programlisting>

        <para>This will fix any errors, and remove unrecoverable files.
        Directories will be recreated if necessary.</para>

        <para><emphasis role="bold">NOTE</emphasis>: The utility may adjust
        the soft and hard limits on the account to make sure that housekeeping
        will not remove anything -- check these afterwards.</para>
      </section>

      <section>
        <title>Grab any files you need with bbackupquery</title>

        <para>At this point, you will have a working store. Every file which
        was on the server, and wasn't corrupt, will be available.</para>

        <para>On the client, use bbackupquery to log in and examine the store.
        (type help at the prompt for instructions). Retrieve any files you
        need, paying attention to any lost+found directories in the root
        directory of the store.</para>

        <para>You can skip this step if you are sure that the client machine
        is fine -- in this case, bbackupd will bring the store up to
        date.</para>
      </section>

      <section>
        <title>Restart bbackupd</title>

        <para>Restart bbackupd on the client machine. The store account will
        be brought up to date, and files in the wrong place will be marked for
        eventual deletion.</para>
      </section>
    </section>

    <section>
      <title>Troubleshooting</title>

      <para>If you are trying to fix a store after your disc has been
      corrupted, see <link linkend="FixCorruptions">Fixing corruptions of
      store data</link>.</para>

      <para>Unfortunately, the error messages are not particularly helpful at
      the moment. This page lists some of the common errors, and the most
      likely causes of them.</para>

      <para>When an error occurs, you will see a message like 'Exception:
      RaidFile/OSFileError (2/8)' either on the screen or in your log files.
      (it is recommended you set up another log file as recommended in the
      server setup instructions.)</para>

      <para>This error may not be particularly helpful, although some do have
      extra information about probable causes. To get further information,
      check the ExceptionCodes.txt file in the root of the distribution. This
      file is generated by the ./configure script, so you will need to have
      run that first.</para>

      <para>Some common causes of exceptions are listed below.</para>

      <para>Please email me with any other codes you get, and I will let you
      know what they mean, and add notes here.</para>

      <section>
        <title>RaidFile (2/8)</title>

        <para>This is found either when running bbstoreaccounts or in the
        bbstored logs.</para>

        <para><emphasis role="bold">Problem</emphasis>: The directories you
        specified in the raidfile.conf are not writable by the _bbstored
        user.</para>

        <para><emphasis role="bold">Resolution</emphasis>: Change permissions
        appropriately.</para>
      </section>

      <section>
        <title>Common (1/2)</title>

        <para>This usually occurs when the configuration files can't be
        opened.</para>

        <para><emphasis role="bold">Problem</emphasis>: You created your
        configurations in non-standard locations, and the programs cannot find
        them.</para>

        <para><emphasis role="bold">Resolution</emphasis>: Explicitly specify
        configuration file locations to daemons and programs. For
        example</para>

        <programlisting>/usr/local/bin/bbstored /some/other/dir/bbstored.config /usr/local/bin/bbackupquery -c /some/other/dir/bbackupd.config</programlisting>

        <para>(daemons specify the name as the first argument, utility
        programs with the -c option).</para>

        <para><emphasis role="bold">Problem</emphasis>: bbstored can't find
        the raidfile.conf file specified in bbstored.conf.</para>

        <para><emphasis role="bold">Resolution</emphasis>: Edit bbstored.conf
        to point to the correct location of this additional configuration
        file.</para>
      </section>

      <section>
        <title>Server (3/16)</title>

        <para>The server can't listen for connections on the IP address
        specified when you configured it.</para>

        <para><emphasis role="bold">Problem</emphasis>: This probably means
        you've specified the wrong hostname to bbstored-config -- maybe your
        server is behind a NAT firewall?</para>

        <para><emphasis role="bold">Resolution</emphasis>: Edit bbstored.conf
        and correct the ListenAddresses line. You should replace the server
        address with the IP address of your machine.</para>
      </section>

      <section>
        <title>Connection (7/x)</title>

        <para>These errors all relate to connections failing -- you may see
        them during operation if there are network failures or other problems
        between the client and server. The backup system will recover from
        them automatically.</para>

        <section>
          <title>Connection (7/30) - SSL problems</title>

          <para>Log snippet from client side:</para>

          <programlisting>bbackupd[1904]: Opening connection to server xxxx.xxx...
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:asn1 encoding routines:ASN1_verify:EVP lib
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
bbackupd[1904]: TRACE: Exception thrown: ConnectionException(Conn_TLSHandshakeFailed) at SocketStreamTLS.cpp(237)
bbackupd[1904]: Exception caught (7/30), reset state and waiting to retry...</programlisting>

          <para>And from the server:</para>

          <programlisting>bbstored[19291]: Incoming connection from xx.xxx.xx.xxx port xxxxx (handling in child xxxxx)
bbstored[21588]: SSL err during Accept: error:xxxxxxxx:SSL routines:SSL3_READ_BYTES:tlsv1 alert decrypt error
bbstored[21588]: in server child, exception Connection TLSHandshakeFailed (7/30) -- terminating child</programlisting>

          <para><emphasis role="bold">Solution</emphasis>: Create a new CA on
          the server side and re-generate the client certificate. Re-creating
          the client certificate request is not necessary.</para>
        </section>
      </section>

      <section>
        <title>Advanced troubleshooting</title>

        <para>If this really doesn't help, then using the DEBUG builds of the
        system will give you much more information -- a more descriptive
        exception message and the file and line number where the error
        occurred.</para>

        <para>For example, if you are having problems with bbstoreaccounts,
        build the debug version with:</para>

        <programlisting>cd boxbackup-0.0
cd bin/bbstoreaccounts
make</programlisting>

        <para>Within the module directories, make defaults to building the
        debug version. At the top level, it defaults to release.</para>

        <para>This will build an executable in debug/bin/bbstoreaccounts which
        you can then use instead of the release version. It will give far more
        useful error messages.</para>

        <para>When you get an error message, use the file and line number to
        locate where the error occurs in the code. There will be comments
        around that line to explain why the exception happened.</para>

        <para>If you are using a debug version of a daemon, these extended
        messages are found in the log files.</para>
      </section>
    </section>
  </chapter>

  &__ExceptionCodes__elfjz3fu;

  <appendix>
    <title id="WORoot">Running without root</title>

    <para>It is possible to run both the server and client without root
    privileges.</para>

    <section>
      <title>Server</title>

      <para>The server, by default, runs as a non-root user. However, it
      expects to be run as root and changes user to a specified user as soon
      as it can, simply for administrative convenience. The server uses a port
      greater than 1024, so it doesn't need root to start.</para>

      <para>To run it entirely as a non-root user, edit the bbstored.conf
      file, and remove the User directive from the Server section. Then simply
      run the server as your desired user.</para>
    </section>

    <section>
      <title>Client</title>

      <para>The client requires root for normal operation, since it must be
      able to access all files to back them up. However, it is possible to run
      the client as a non-root user, with certain limitations.</para>

      <para>Follow the installation instructions, but install the executable
      files manually to somewhere in your home directory. Then use
      bbackupd-config to configure the daemon, but use a directory other than
      /etc/box, probably somewhere in your home directory.</para>

      <para>All directories you specify to be backed up must be readable, and
      all files must be owned by the user and readable to that user.</para>

      <para>Important: If any file or directory is not readable by this user,
      the backup process will skip that file or directory. Keep an eye on the
      logs for reports of this failure.</para>

      <para>Non-root operation of the backup client is recommended only for
      testing, and should not be relied on in a production environment.</para>
    </section>
  </appendix>
</book>