Box Backup administrator's guideLicenseCopyright (c) <YEAR>, <OWNER>All rights reserved.Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.Neither the name of the <ORGANIZATION> nor the names of
its contributors may be used to endorse or promote products derived
from this software without specific prior written permissionTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.ConfigurationSystem configurationServerAfter you've downloaded and compiled the programs you need to
install the programs on your server. As root do the following:make install-backup-serverThis assumes that you are installing on the same server that you
compiled the software on. If not, copy the
boxbackup-x.xx-backup-server-OSNAME.tgz file to the server you want to
run on, and install there. For example (on Mac OS X):tar zxvf boxbackup-0.10-server-darwin8.5.0.tgz
cd boxbackup-0.10-server-darwin8.5.0
./install-backup-serverThen create the user for the backup daemon on the server:useradd _bbstoredBoxBackup supports Raid for the backup store. There are some
configuration options. Use the following command if you want to create
a simple server WITHOUT Raid protection:mkdir /tmp/boxbackupRepository # Create the directory
chown _bbstored /tmp/boxbackupRepository/ # Change the owner to the new boxbackup daemon user
/usr/local/bin/raidfile-config /etc/box/ 1024 /tmp/boxbackupRepository
#substitute 1024 with the desired blocksize
#substitute /tmp/boxbackupRepository with a directory that exists where you want the backup store located
#/usr/local/bin/raidfile-config --help shows you the optionsThen create the configuration file /etc/box/bbstored.conf The
hostname is tricky as it is used for two things: The name of the
server in the certificate and the address the server is listening on.
Since you might be using NAT, might move the server around or the
domain name might change, choose a name that describes the server.
When the network address of the server changes, you need to update the
ListenAddresses directive in the /etc/box/bbstored.conf file./usr/local/bin/bbstored-config /etc/box hostname _bbstoredThis last step outputs 5 instructions that you must execute to
the letter. A lot of questions are raised on the mailing list because
these steps have not been followed properly.TODO: Expand on this. Explain the 5 steps in detail.If you want to run the system as a non-root user, look here.Certificate ManagementThere are two steps involved to create an account. You need to
create the account on the server, and sign a certificate to give the
client permission to connect to the server.Running a Certification Authority for TLS (SSL) connections is
not trivial. However, a script to does most of the work in a way which
should be good enough for most deployments.Important Note: The certificate
authority directory is intended to be stored on another server. It
should not be kept on the backup server to limit the impact of a
server compromise. The instructions and the script assume that it will
be kept elsewhere, so will ask you to copy files to and from the CA.
Clock time warning: SSL
certificates contain validity dates, including a "valid from" time. If
the clock on the machine which signs the certificates is not
syncronised to the clocks of the machines using these certificates,
you will probably get strange errors until the start time is reached
on all machines. If you get strange errors when attempting to use new
certificates, check the clocks! You will probably just need to wait a
while until the certificates become valid, rather than having to
regenerate them. Set up a CAIt's best to do this on a machine other than your server,
probably without direct network access. The contents of this
directory control who can access your backup store server.To setup the basic key structure, do the following:/usr/local/bin/bbstored-certs ca init(See OpenSSL notes if
you get an OpenSSL error)This creates the directory 'ca' in the current directory, and
initialises it with basic keys. Sign a server certificateWhen you use the bbstored-config script to set up a config
file for a server, it will generate a certificate request (CSR) for
you. Transfer it to the machine with your CA, then do:/usr/local/bin/bbstored-certs ca sign-server hostname-csr.pemThis signs the certificate for the server. Follow the
instructions in the output on which files to install on the server.
The CSR file is now no longer needed. Make sure you run this command
from the directory above the directory 'ca'. TODO: Explain instructions in output.Set up an accountChoose an account number for the user. This must be unique on
the server, and is presented as a 31 bit number in hex greater than
0, for example, 1 or 75AB23C. Then on the backup store server,
create the account with:/usr/local/bin/bbstoreaccounts create 75AB23C 0 4096M 4505MThis looks complicated. The numbers are, in order:The account number allocated (hex)The RAID disc set (0 if you use raidfile-config and don't
add a new set)Soft limit (size)Hard limit (size)The sizes are are specified in Mb, Gb, or blocks, depending on
the suffix. 1M specifies 1 Mb, 1G specifies 1 Gb, and 1B specifies 1
block, the size of which depends on how you have configured the
raidfile system with raidfile-config.In this example, I have allocated 4Gb (assuming you use 2048
byte blocks as per my example) as the soft limit, and 4Gb + 10% as
the hard limit.NOTE The sizes specified here are pre-RAID. So if you are
using userland RAID, you are actually allocating two-thirds of this
amount. This means that, when you take compression into account,
that if you allocate 2Gb on the server, it'll probably hold about
2Gb of backed up files (depending on the compressability of those
files).The backup client will (voluntarily) try not to upload more
data than is allowed by the soft limit. The store server will refuse
to accept a file if it would take it over the hard limit, and when
doing housekeeping for this account, try and delete old versions and
deleted files to reduce the space taken to below the soft
limit.This command will create some files on disc in the raid file
directories (if you run as root, the utility will change to the user
specified in the bbstored.conf file to write them) and update the
accounts file. A server restart is not required.NOTE If you get a message saying 'Exception: RaidFile (2/8)',
the directories you specified in the raidfile.conf are not writable
by the _bbstored user -- fix it, and try again.Finally, tell the user their account number, and the hostname
of your server. They will use this to set up the backup client, and
send you a CSR. This has the account number embedded in it, and you
should be sure that it has the right account number in it.Sign this CSR with this command:/usr/local/bin/bbstored-certs ca sign 75AB23C-csr.pemDon't forget to check that the embedded account number is
correct! Then send the two files back to the user, as instructed by
the script.Please read the Troubleshooting page if you have problems.
TODO: Link to troubleshooting...Log FilesYou may wish to see what's going on with the server. Edit
/etc/syslog.conf, and add:local6.info /var/log/box
local5.info /var/log/raidfileNote: Separators must be tabs,
otherwise these entries will be ignored.touch /var/log/box
touch /var/log/raidfileSet up log rotation, by adding in /etc/newsyslog.conf:/var/log/box 644 7 2000 * Z
/var/log/raidfile 644 7 2000 * ZThen restart syslogd.Configuring a clientBefore you can do any configuration, you need to know the
hostname of the server you will be using, and your account number on
that server.Later in the process, you will need to send a certificate
request to the administrator of that server for it to be
signed.Installation is covered in the compiling and installing section.
You only need the backup-client parcel.It is important that you read all the output of the config
scripts. See the end of this page for an example.The backup client has to be run as root, because it needs to
read all your files to back them up, although it is possible to back
up a single user's files by running it as that user. (Tip: specify a
directory other than /etc/box, and then give the alternate config file
as the first argument to bbackupd). However, it will fall over if you
don't give yourself read access to one of your files. Basic configurationRun the bbackupd-config script to generate the configuration
files and generate a private key and certificate request./usr/local/bin/bbackupd-config /etc/box lazy 999 hostname /var/bbackupd /home(See OpenSSL notes if
you get an OpenSSL error)The items in bold need to be changed. In order, they are the
account number, the hostname of the server you're using, and
finally, the directories you want backed up. You can include as many
you want here.However, the directories you specify must not contain other
mounted file systems within them at any depth. Specify them
separately, one per mount point. No checks are currently made to
catch bad configuration of this nature!You may also want to consider changing the mode from lazy to
snapshot, depending on what your system is used for. lazy. This mode regularly
scans the files, with only a rough schedule. It uploads files as
and when they are changed, if the latest version is more than a
set age. This is good for backing up user's documents stored on
a server, and spreads the load out over the day. snapshot. This mode
emulates the traditional backup behaviour of taking a snapshot
of the filesystem. The backup daemon does absolutely nothing
until it is instructed to make a backup using the bbackupctl
utility (probably as a cron job), at which point it uploads all
files which have been changed since the last time it uploaded.
When you run the config script, it will tell you what you need
to do next. Don't forget to read all the output. An example is shown
at the end of this page, but the instructions for your installation
may be different.CertificatesAfter you have sent your certificate request off to the server
administrator and received your certificate and CA root back,
install them where instructed by the bbackupd-config script during
basic bbackupd configuration.You can then run the daemon (as root) by typing
/usr/local/bin/bbackupd, and of course, adding it to your system's
startup scripts. The first time it's run it will upload everything.
Interrupting it and restarting it will only upload files which were
not uploaded before - it's very tolerant.If you run in snapshot mode, you will need to add a cron job
to schedule backups. The config script will tell you the exact
command to use for your system.Please read the Troubleshooting page if you have
problems.Remember to make a traditional backup of the keys file, as
instructed. You cannot restore files without it.It is recommended that you backup up all of /etc/box as it
will make things easier if you need to restore files. But only the
keys are absolutely essential.If you want to see what it's doing in more detail (probably a
good idea), follow the instructions in the server setup to create
new log files with syslog. Adding and removing backed up
locations.By editing the /etc/box/bbackupd.conf file, you can add and
remove directories to back up -- see comments in this file for help.
Send bbackupd a HUP signal after you modify it.When you remove a location, it will not be marked as deleted
immediately. Instead, bbackupd waits about two days before doing so,
just in case you change your mind. After this, it will be eventually
removed from the store by the housekeeping process. Run as
root.The backup client is designed to be run as root. It is
possible to run without root, but this is not recommended. Clock
synchronisation for file servers.If you are using the backup client to backup a filesystem
served from a fileserver, you should ideally ensure that the
fileserver clocks are synchronised with the fileserver.bbackupd will cope perfectly well if the clocks are not
synchronised. Errors up to about half an hour cause no problems.
Larger discrepancies cause a loss of efficiency and the potential to
back up a file during a write process.There is a configuration parameter MaxFileTimeInFuture, which
specifies how far in the future a file must be for it to be uploaded
as soon as it is seen. You should not need to adjust this (default
is 2 days). Instead, get those clocks synchronised. Excluding files
and directories from the backup.Within the bbackupd.conf file, there is a section named
BackupLocations which specifies which locations on disc should be
backed up. It has subsections, each of which is in the
format: name
{
Path = /path/of/directory
(optional exclude directives)
}name is derived from the Path
by the config script, but should merely be unique.The exclude directives are of the form:[Exclude|AlwaysInclude][File|Dir][|sRegex] = regex or full pathname(The regex suffix is shown as 'sRegex' to make File or Dir
plural)For example: ExcludeDir = /home/guest-user
ExcludeFilesRegex = *.(mp3|MP3)\$
AlwaysIncludeFile = /home/username/veryimportant.mp3This excludes the directory /home/guest-user from the backup
along with all mp3 files, except one MP3 file in particular.In general, Exclude excludes a file or directory, unless the
directory is explicitly mentioned in a AlwaysInclude
directive.If a directive ends in Regex, then it is a regular expression
rather than a explicit full pathname. See man 7 re_formatfor the regex syntax on your platform.Example configuration outputThis is an example of output from the bbstored-config
script.Important: Follow the
instructions output by your script, not the ones here -- they may be
different for your system./usr/local/bin/bbackupd-config /etc/box lazy 51 server.example.com /var/bbackupd /home /etc/samba
Setup bbackupd config utility.
Configuration:
Writing configuration file: /etc/box/bbackupd.conf
Account: 51
Server hostname: server.example.com
Directories to back up:
/home
/etc/samba
Note: If other file systems are mounted inside these directories, then problems may occur
with files on the store server being renamed incorrectly. This will cause efficiency
problems, but not affect the integrity of the backups.
WARNING: Directories not checked against mountpoints. Check mounted filesystems manually.
Creating /etc/box...
Creating /etc/box/bbackupd
Generating private key...
[OpenSSL output omitted]
Generating keys for file backup
Writing notify script /etc/box/bbackupd/NotifyStoreFull.sh
Writing configuration file /etc/box/bbackupd.conf
===================================================================
bbackupd basic configuration complete.
What you need to do now...
1) Make a backup of /etc/box/bbackupd/51-FileEncKeys.raw
This should be a secure offsite backup.
Without it, you cannot restore backups. Everything else can
be replaced. But this cannot.
KEEP IT IN A SAFE PLACE, OTHERWISE YOUR BACKUPS ARE USELESS.
2) Send /etc/box/bbackupd/51-csr.pem
to the administrator of the backup server, and ask for it to
be signed.
3) The administrator will send you two files. Install them as
/etc/box/bbackupd/51-cert.pem
/etc/box/bbackupd/serverCA.pem
after checking their authenticity.
4) You may wish to read the configuration file
/etc/box/bbackupd.conf
and adjust as appropraite.
There are some notes in it on excluding files you do not
wish to be backed up.
5) Review the script
/etc/box/bbackupd/NotifyStoreFull.sh
and check that it will email the right person when the store
becomes full. This is important -- when the store is full, no
more files will be backed up. You want to know about this.
6) Start the backup daemon with the command
/usr/local/bin/bbackupd
in /etc/rc.local, or your local equivalent.
Note that bbackupd must run as root.
===================================================================Remember to make a secure, offsite backup of your backup keys,
as described in Basic
configuration above. If you do not,, and that key is lost,
you have no backups.AdministrationThis chapter deals with the dauily running and management of the Box
Backup system. It explains most day-to-day tasks.Regular MaintenanceThe steps involved in maintaining and keeping the backup sets
healthy are outlined in this section.Controlling a backup clientThe bbackupctl program sends control commands to the bbackupd
daemon. It must be run as the same user as the daemon, and there is no
exception for root.The command line syntax is:/usr/local/bin/bbackupctl [-q] [-c config-file] commandThe -q option reduces the amount of output the program emits,
and -c allows an alternative configuration file to be
specified.Valid commands are: terminateStop the bbackupd daemon now (equivalent to kill)reloadReload the configuration file (equivalent to kill
-HUP)syncConnect to the server and synchronise files nowbbackupctl communicates with
the server via a UNIX domain socket, specified in bbackupd.conf with
the CommandSocket directive. This does not need to be specified, and
bbackupd will run without the command
socket, but in this case bbackupctl will not be able to communicate
with the daemon.Some platforms cannot check the user id of the connecting
process, so this command socket becomes a denial of service security
risk. bbackupd will warn you when it
starts up if this is the case on your platform, and you should
consider removing the CommandSocket directive on these
platforms.Using bbackupctl to perform snapshotsbbackupctl's main purpose is to
implement snapshot based backups, emulating the behaviour of
traditional backup software.Use bbackupd-config to write a configuration file in snapshot
mode, and then run the following command as a cron job. /usr/local/bin/bbackupctl -q syncThis will cause the backup daemon to upload all changed files
immediately. bbackupctl will exit
almost immediately, and will not output anything unless there is an
error. Checking storage space used on the serverFrom the client machinebbackupquery can tell you how much space is used on the server
for this account. Either use the usage command in interactive mode,
or type:/usr/local/bin/bbackupquery -q usage quitto show the space used as a single command.On the serverbbstoreaccounts allows you to query the space used, and change
the limits. To display the space used on the server for an account,
use:/usr/local/bin/bbstoreaccounts info 75AB23CTo adjust the soft and hard limits on an account, use:/usr/local/bin/bbstoreaccounts setlimit 75AB23C new-soft-limit new-hard-limitYou do not need to restart the server.Verify and restore filesBackups are no use unless you can restore them. The bbackupquery
utility does this and more.You don't provide any login information to it, as it just picks
up the data it needs from /etc/box/bbackupd.conf. You should run it as
root so it can find everything it needs.Full documentation can be found in the bbackupquery man page. It follows
the model of a command line sftp client quite closely.TODO: Link to bbackupquery man-page here.On systems where GNU readline is available (by default) it uses
that for command line history and editing. Otherwise it falls back to
very basic UNIX text entry. TODO: Did the readline dependency change to editline?Using bbackupquerybbackupquery is the tool you use to verify, restore and
investigate your backup files with. When invoked, it simply logs
into the server using the certificates you have listed in
bbackupd.conf. After you run bbackupquery, you will see a prompt, allowing
you to execute commands. The list (or ls) command lets you view
files in the store. It works much like unix ls, but with different
options. An example:[pthomsen@host bbackupquery]$ bbackupquery
Box Backup Query Tool v0.10, (c) Ben Summers and contributors 2003-2006
Using configuration file /etc/box/bbackupd.conf
Connecting to store...
Handshake with store...
Login to store...
Login complete.
Type "help" for a list of commands.
query > ls
00000002 -d---- mp3
00000003 -d---- video
00000004 -d---- home-pthomsen
00000005 -d---- root
query > The ls commands shows the directories that are backed up. Now
we'll take a closer look at the home-pthomsen directory:query > cd home-pthomsen
query > ls
00002809 f----- sample.tiff
0000280a f----- s3.tiff
0000280b f----- s4.tiff
0000280d f----- s2.tiff
0000280e f----- foo.pdf
0000286c f----- core.28720
0000339a -d---- .emacs.d
0000339d -d---- bbackup-contrib
00003437 f----- calnut.compare.txt
0000345d f----- DSCN1783.jpg
0000345e f----- DSCN1782.jpg
query >The ls command takes the following options;-r -- recursively list
all files-d -- list deleted
files/directories-o -- list old versions
of files/directories-I -- don't display
object ID-F -- don't display
flags-t -- show file
modification time (and attr mod time if has the object has
attributes, ~ separated)-s -- show file size in
blocks used on server (only very approximate indication of size
locally)The flags displayed from the ls command are as follows:f = filed = directoryX = deletedo = old versionR = remove from server as soon as marked deleted or
olda = has attributes stored in directory record which
override attributes in backup fileVerify backupsSince this system is not yet 100% production-ready, you'll be
keen to verify that your backups are correct. This is easy:/usr/local/bin/bbackupquery "compare -a" quitIt will report all the differences between the store and the
files on disc. It will download everything, so may take a while. You
should expect to see some differences on a typical compare, because
files which have recently changed are unlikely to have been uploaded
yet. Consider checking the timestamps on the files, or keeping a
record of these messages and comparing them with a future
verification.If you would like to do a "quick" check which just downloads
file checksums and compares against that, then do:/usr/local/bin/bbackupquery "compare -aq" quitHowever, this does not check that the file attributes are
correct, and since the checksums are generated on the client they
may not reflect the data on the server if there is a problem -- the
server cannot check the encrypted contents. View this as a good
indication, rather than a definite check that your backup verifies
correctly.You may wish to run either one as a cron job while testing
this system. Restore backupsYou will need the keys file created when you configured the
server. Without it, you cannot restore the files; this is the
downside of encrypted backups. However, by keeping the small keys
file safe, you indirectly keep your entire backup safe.The first step is to recreate the configuration of the backup
client. It's probably best to have stored the /etc/box directory
with your keys. But if you're recreating it, all you really need is
to have got the login infomation correct (ie the certs and
keys).Don't run bbackupd yet! It will mark all your files as deleted
if you do, which is not hugely bad in terms of losing data, just a
major inconvenience. (This assumes that you are working from a blank
slate. If you want to restore some files to a different location,
it's fine to restore while bbackupd is running, just do it outside a
backed up directory to make sure it doesn't start uploading the
restored files.)Type: /usr/local/bin/bbackupqueryto run it in interactive mode.Type:listto see a list of the locations stored on the server.For each location you want to restore, type:restore name-on-server local-dir-nameThe directory specified by local-dir-name must not exist yet.
If the restore is interrupted for any reason, repeat the above
steps, but add the -r flag to the
restore command to tell it to resume.Retrieving deleted and old filesBox Backup makes old versions of files and files you have
deleted available, subject to there being enough disc space on the
server to hold them.This is how to retrieve them using bbackupquery. Future
versions will make this far more user-friendly.Firstly, run bbackupquery in interactive mode. It behaves in a
similar manner to a command line sftp client. /usr/local/bin/bbackupqueryThen navigate to the directory containing the file you want,
using list, cd and pwd.query > cd home/profiles/USERNAMEList the directory, using the "o" option to list the files
available without filtering out everything apart from the current
version. (if you want to see deleted files as well, use list
-odt)query > list -ot
00000078 f--o- 2004-01-21T20:17:48 NTUSER.DAT
00000079 f--o- 2004-01-21T20:17:48 ntuser.dat.LOG
0000007a f--o- 2004-01-21T17:55:12 ntuser.ini
0000007b f---- 2004-01-12T15:32:00 ntuser.pol
0000007c -d--- 1970-01-01T00:00:00 Templates
00000089 -d--- 1970-01-01T00:00:00 Start Menu
000000a0 -d--- 1970-01-01T00:00:00 SendTo
000000a6 -d--- 1970-01-01T00:00:00 Recent
00000151 -d--- 1970-01-01T00:00:00 PrintHood
00000152 -d--- 1970-01-01T00:00:00 NetHood
00000156 -d--- 1970-01-01T00:00:00 My Documents
0000018d -d--- 1970-01-01T00:00:00 Favorites
00000215 -d--- 1970-01-01T00:00:00 Desktop
00000219 -d--- 1970-01-01T00:00:00 Cookies
0000048b -d--- 1970-01-01T00:00:00 Application Data
000005da -d--- 1970-01-01T00:00:00 UserData
0000437e f--o- 2004-01-24T02:45:43 NTUSER.DAT
0000437f f--o- 2004-01-24T02:45:43 ntuser.dat.LOG
00004380 f--o- 2004-01-23T17:01:29 ntuser.ini
00004446 f--o- 2004-01-24T02:45:43 NTUSER.DAT
00004447 f--o- 2004-01-24T02:45:43 ntuser.dat.LOG
000045f4 f---- 2004-01-26T15:54:16 NTUSER.DAT
000045f5 f---- 2004-01-26T15:54:16 ntuser.dat.LOG
000045f6 f---- 2004-01-26T16:54:31 ntuser.ini(this is a listing from a server which is used as a Samba
server for a network of Windows clients.) You now need to fetch the
file using it's ID, rather than it's name. The ID is the hex number
in the first column. Fetch it like this: query > get -i 0000437e NTUSER.DAT
Object ID 0000437e fetched successfully.The object is now available on your local machine. You can use
lcd to move around, and sh ls to list directories on your local
machine.Fixing corruptions of store dataThis section gives help on what to do if your server has suffered
corruption, for example, after an unclean shutdown or other OS or
hardware problem.In general, as updates to the store are made in an atomic manner,
the most likely result is wasted disc space. However, if really bad
things happen, or you believe that there is a lot of wasted space, then
these instructions will help to restore your data.You know you will need to do something if you get strange errors,
and bbackupd attempts to contact the server every 100 seconds or so. Or
if one of the discs in your RAID disc set has failed.After following these instructions, the end result will be that
bbackupquery will be able to see all the files which were stored on your
server, and retrieve them. Some of them may be in lost+found directories
in the root of the store (or in their original position if they have
been moved) but they will all be able to be retrieved.After you have retrieved the files you want, bbackupd will upload
new versions where necessary, and after about two days, mark any
lost+found directories as deleted. Finally, those directories will be
removed by the housekeeping process on the server.These instructions assume you're working on account 1234,
subsitute this for whatever account you're actually working on. These
will need to be repeated for all affected accounts. Stop bbackupdFirst, make sure that bbackupd is not running on the client
machine for the account you are going to recover. Use kill to
terminate it. This step is not strictly necessary, but is recommended.
During any checks on the account, bbackupd will be unable to log in,
and after they are complete, the account is marked as changed on the
server so bbackupd will perform a complete scan.Are you using RAID on the server?At the moment, the raidfile recovery tools have not been
written. However, when two out of three files are available, the
server will run succesfully, even if it complains a lot in the logs.
So, your best bet here is to fix the accounts, if necessary, and
retrieve any files you need. Then move the old store directories aside
(in case you need them) and start afresh with new accounts, and let
the clients upload all their data again. These utilities will be
written shortly! TODO: Is this true anymore???Check and fix the accountFirst, run the check utility, and see what errors it
reports./usr/local/bin/bbstoreaccounts check 1234This will take some time, and use a fair bit of memory (about 16
bytes per file and directory). If the output looks plausible and
reports errors which need fixing, run it again but with the fix
flag:/usr/local/bin/bbstoreaccounts check 1234 fixThis will fix any errors, and remove unrecoverable files.
Directories will be recreated if necessary.NOTE: The utility may adjust
the soft and hard limits on the account to make sure that housekeeping
will not remove anything -- check these afterwards. Grab any files you need with bbackupqueryAt this point, you will have a working store. Every file which
was on the server, and wasn't corrupt, will be available.On the client, use bbackupquery to log in and examine the store.
(type help at the prompt for instructions). Retrieve any files you
need, paying attention to any lost+found directories in the root
directory of the store.You can skip this step if you are sure that the client machine
is fine -- in this case, bbackupd will bring the store up to date.
Restart bbackupdRestart bbackupd on the client machine. The store account will
be brought up to date, and files in the wrong place will be marked for
eventual deletion.TroubleshootingIf you are trying to fix a store after your disc has been
corrupted, see Fixing corruptions of
store data.Unfortunately, the error messages are not particularly helpful at
the moment. This page lists some of the common errors, and the most
likely causes of them.When an error occurs, you will see a message like 'Exception:
RaidFile/OSFileError (2/8)' either on the screen or in your log files.
(it is recommended you set up another log file as recommended in the
server setup instructions.)This error may not be particularly helpful, although some do have
extra information about probable causes. To get further information,
check the ExceptionCodes.txt file in the root of the distribution. This
file is generated by the ./configure script, so you will need to have
run that first.Some common causes of exceptions are listed below.Please email me with any other codes you get, and I will let you
know what they mean, and add notes here. RaidFile (2/8)This is found either when running bbstoreaccounts or in the
bbstored logs.Problem: The directories you
specified in the raidfile.conf are not writable by the _bbstored
user.Resolution: Change permissions
appropriately.Common (1/2)This usually occurs when the configuration files can't be
opened.Problem: You created your
configurations in non-standard locations, and the programs cannot find
them.Resolution: Explicitly specify
configuration file locations to daemons and programs. For
example/usr/local/bin/bbstored /some/other/dir/bbstored.config /usr/local/bin/bbackupquery -c /some/other/dir/bbackupd.config(daemons specify the name as the first argument, utility
programs with the -c option).Problem: bbstored can't find
the raidfile.conf file specified in bbstored.conf.Resolution: Edit bbstored.conf
to point to the correct location of this additional configuration
file. Server (3/16)The server can't listen for connections on the IP address
specified when you configured it.Problem: This probably means
you've specified the wrong hostname to bbstored-config -- maybe your
server is behind a NAT firewall?Resolution: Edit bbstored.conf
and correct the ListenAddresses line. You should replace the server
address with the IP address of your machine. Connection (7/x)These errors all relate to connections failing -- you may see
them during operation if there are network failures or other problems
between the client and server. The backup system will recover from
them automatically.Connection (7/30) - SSL problemsLog snippet from client side:bbackupd[1904]: Opening connection to server xxxx.xxx...
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:asn1 encoding routines:ASN1_verify:EVP lib
bbackupd[1904]: SSL err during Connect: error:xxxxxxxx:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
bbackupd[1904]: TRACE: Exception thrown: ConnectionException(Conn_TLSHandshakeFailed) at SocketStreamTLS.cpp(237)
bbackupd[1904]: Exception caught (7/30), reset state and waiting to retry...And from the server:bbstored[19291]: Incoming connection from xx.xxx.xx.xxx port xxxxx (handling in child xxxxx)
bbstored[21588]: SSL err during Accept: error:xxxxxxxx:SSL routines:SSL3_READ_BYTES:tlsv1 alert decrypt error
bbstored[21588]: in server child, exception Connection TLSHandshakeFailed (7/30) -- terminating childSolution: Create a new CA on
the server side and re-generate the client certificate. Re-creating
the client certificate request is not necessary.Advanced troubleshootingIf this really doesn't help, then using the DEBUG builds of the
system will give you much more information -- a more descriptive
exception message and the file and line number where the error
occurred.For example, if you are having problems with bbstoreaccounts,
build the debug version with:cd boxbackup-0.0
cd bin/bbstoreaccounts
makeWithin the module directories, make defaults to building the
debug version. At the top level, it defaults to release.This will build an executable in debug/bin/bbstoreaccounts which
you can then use instead of the release version. It will give far more
useful error messages.When you get an error message, use the file and line number to
locate where the error occurs in the code. There will be comments
around that line to explain why the exception happened.If you are using a debug version of a daemon, these extended
messages are found in the log files.Running without rootIt is possible to run both the server and client without root
privileges.ServerThe server, by default, runs as a non-root user. However, it
expects to be run as root and changes user to a specified user as soon
as it can, simply for administrative convenience. The server uses a port
greater than 1024, so it doesn't need root to start.To run it entirely as a non-root user, edit the bbstored.conf
file, and remove the User directive from the Server section. Then simply
run the server as your desired user. ClientThe client requires root for normal operation, since it must be
able to access all files to back them up. However, it is possible to run
the client as a non-root user, with certain limitations.Follow the installation instructions, but install the executable
files manually to somewhere in your home directory. Then use
bbackupd-config to configure the daemon, but use a directory other than
/etc/box, probably somewhere in your home directory.All directories you specify to be backed up must be readable, and
all files must be owned by the user and readable to that user.Important: If any file or directory is not readable by this user,
the backup process will skip that file or directory. Keep an eye on the
logs for reports of this failure.Non-root operation of the backup client is recommended only for
testing, and should not be relied on in a production environment.