path: root/debian/testing-notes.org

#+TITLE: boxbackup testing notes
#+DATE: <2017-06-17 Sat>
#+AUTHOR: Reinhard Tartler
#+EMAIL: siretart@debian.org

* Installing vagrant

These are my personal notes on how I've tested the boxbackup
packages. Surely there are many other ways (and even better ones) how
test the packages, but this is a way that works for me. Please do
provide me with your thoughts and suggestions.

I'm using vagrant in KVM mode, which is included in debian/stretch. I've
tested this on a relatively modern Intel I5-7260U CPU with a Debian
amd64 installation. Other architectures and non-virtualized environments
should work exactly the same.  The basic testing idea is to install the
CA signature server, the boxbackup server and the boxbackup client on
the same (virtual) machine, and backup to localhost for
simplicity. Production installations will place them all on different
host, but that would make testing unnecessarily hard.

Let's first start with installing vagrant:

#+BEGIN_SRC bash
sudo apt install vagrant-libvirt virt-manager
sudo adduser $(whoami) libvirt
newgrp
#+END_SRC

Now we can get a new box, get it up and login:

#+BEGIN_SRC bash
vagrant init debian/stretch64
vagrant up
vagrant ssh
#+END_SRC

You might get some password prompts, not sure how to avoid those.

* Testing boxbackup

First install the debian packages. This assumes that the Vagrant file is
in the same directory that contains the =*.deb= packages to test.

#+BEGIN_SRC bash
sudo apt install /vagrant/boxbackup*0.13*.deb
#+END_SRC

First, we need to create a certificate authority.

#+BEGIN_SRC bash
cd /root
bbstored-certs ca init
#+END_SRC

Setup the server:

#+BEGIN_SRC bash
mkdir /boxbackup/
dpkg-reconfigure -p low boxbackup-server
#+END_SRC

Output might look like this:

#+BEGIN_EXAMPLE
Configuring boxbackup-server
----------------------------

The package configuration scripts can create the configuration files for the BoxBackup server.

You should choose this option if you are not familiar with BoxBackup's configuration options. The configuration can be done manually with 
the 'raidfile-config' and 'bbstored-config' scripts.

The server will not start if it is not configured. In all cases, reading the /usr/share/doc/boxbackup-server/README.Debian is recommended.

Should BoxBackup be configured automatically? [yes/no] 


Should BoxBackup be configured automatically? [yes/no] yes
yes


Please choose the location for the three RAID file directories.

To enable RAID, the directory names should be a space-separated list of three partitions, each on different physical hard drives (for 
example: '/raid/0.0 /raid/0.1 /raid/0.2').

If you don't want to enable RAID, just specify the path to one directory where the backups will be stored (for example, 
/usr/local/lib/boxbackup).

These directories will be created if they do not exist.

Location of the RAID directories: /boxbackup/0
/boxbackup/0


BoxBackup uses userland RAID techniques.

Please choose the block size to use for the storage. For maximum efficiency, you should choose the block size of the underlying file 
system (which can be displayed for ext2 filesystems with the 'tune2fs -l' command).

This value should be set even if you don't plan to use RAID.

Block size for the userland RAID system: 4096
4096


The BoxBackup server needs an RSA private key and the corresponding X.509 certificate to perform client-server authentication and 
communication encryption.

Both can be generated automatically. You will need to sign the certificate with your root CA (see the boxbackup-server package) and put 
this signed certificate and the root CA certificate in the configuration folder.

Generate a server private key and X.509 certificate request? [yes/no] yes
yes


User bbstored already exists.
Creating /boxbackup/0/backup directory...
Generating RSA private key, 2048 bit long modulus
...................+++
............+++
e is 65537 (0x010001)
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:State or Province Name (full name) [Some-State]:Locality Name (eg, city) []:Organization Name (eg, company) [Internet Widgits Pty Ltd]:Organizational Unit Name (eg, section) []:Common Name (e.g. server FQDN or YOUR name) []:Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:An optional company name []:perl: warning: Setting locale failed.

Creating config file /etc/boxbackup/raidfile.conf with new version

Creating config file /etc/boxbackup/bbstored.conf with new version

#+END_EXAMPLE

Now we need to sign the server certificate:

#+BEGIN_EXAMPLE
cd /root
bbstored-certs ca sign-server /etc/boxbackup/bbstored/boxbackup-server-cert-req.pem

This certificate is for backup server

   localhost

Signing the wrong certificate compromises the security of your backup system.

Would you like to sign this certificate? (type 'yes' to confirm)
yes
yes
Signature ok
subject=CN = localhost
Getting CA Private Key


Certificate signed.

Install the files

   ca/servers/localhost-cert.pem
   ca/roots/clientCA.pem

on the server.

#+END_EXAMPLE

After this, we need to install them:

#+BEGIN_SRC bash
cp -v ca/roots/clientCA.pem /etc/boxbackup/bbstored/boxbackup-client-ca-cert.pem
cp -v ca/servers/localhost-cert.pem /etc/boxbackup/bbstored/boxbackup-server-cert.pem
#+END_SRC

Create a new user:

#+BEGIN_SRC bash
bbstoreaccounts create 1 0 1G 2G
#+END_SRC

Now we can start the server:

#+BEGIN_EXAMPLE
# systemctl restart boxbackup-server
# systemctl status boxbackup-server
● boxbackup-server.service - Box Backup Server
   Loaded: loaded (/lib/systemd/system/boxbackup-server.service; disabled; vendor preset: enabled)
   Active: active (running) since Sat 2017-06-17 23:59:32 UTC; 2s ago
 Main PID: 2574 (bbstored)
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/boxbackup-server.service
           ├─2574 /usr/sbin/bbstored -F -c /etc/boxbackup/bbstored.conf
           └─2575 /usr/sbin/bbstored -F -c /etc/boxbackup/bbstored.conf

Jun 17 23:59:32 stretch systemd[1]: Started Box Backup Server.
Jun 17 23:59:32 stretch bbstored[2574]: NOTICE: Box Backup Store Server v0.12~gitcf52058f-1, (c) Ben Summers and contributors 2003-2014
Jun 17 23:59:32 stretch bbstored[2574]: NOTICE: Starting daemon, version: 0.12~gitcf52058f-1
Jun 17 23:59:32 stretch bbstored[2574]: NOTICE:  Starting daemon, version: 0.12~gitcf52058f-1
Jun 17 23:59:32 stretch bbstored[2574]: NOTICE: Using configuration file: /etc/boxbackup/bbstored.conf
Jun 17 23:59:32 stretch bbstored[2574]: NOTICE:  Using configuration file: /etc/boxbackup/bbstored.conf
#+END_EXAMPLE


Let's create setup the client:

#+BEGIN_SRC bash
# dpkg-reconfigure -plow boxbackup-client
dpkg-reconfigure -plow boxbackup-client
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Configuring boxbackup-client
----------------------------

The package configuration scripts can create the configuration files for the BoxBackup client.

You should choose this option if you are not familiar with BoxBackup's configuration options.

Please read the /usr/share/doc/boxbackup-client/README.Debian for details about the configuration of the BoxBackup client.

Should the BoxBackup client be configured automatically? [yes/no] yes
yes


The BoxBackup client supports two modes of backup:

In the 'lazy' mode, the backup daemon will regularly scan the file system searching for modified files. It will then upload the files 
older than a specified age to the backup server.

In the 'snapshot' mode the backup will be explicitly run at regular intervals. A cron file (/etc/cron.d/boxbackup-client) is provided with
the package and should be adapted to suit your needs.

  1. lazy  2. snapshot

Run mode for the BoxBackup client: 2
2


The administrator of the BoxBackup server should have assigned this client a hexadecimal account number.

If no account number has been assigned yet, leave this field blank and configure it later by running 'dpkg-reconfigure boxbackup-client' 
as root.

Account number for this node on the backup server: 1
1


Please enter the fully qualified domain name of the BoxBackup server which your client will use.

The client will connect to the server on TCP port 2201.

Fully qualified domain name of the backup server: localhost
localhost


Please give a space-separated list of directories to be backed up onto the remote server.

Those directories should not contain mounted file systems at any level in their subdirectories.

List of directories to backup: /etc /home
/etc /home


The BoxBackup client sends alert notifications when a problem occurs during the backup.

Please enter either a local user name (for example 'root') or an email address (for example 'admin@example.org').

Recipient for alert notifications: root
root


The BoxBackup client needs an RSA private key and the corresponding X.509 certificate to authenticate itself with the server.

Both can be generated automatically. You will need to send the certificate request to the BoxBackup server administrator who will sign it 
and send it back to you along with the server's Certification Authority certificate.

These files should be copied into BoxBackup's configuration directory. The file names to use are given in the /etc/boxbackup/bbackupd.conf
file.

Generate the client private key and X.509 certificate request? [yes/no] yes
yes
#+END_SRC

Which we can now sign:



#+BEGIN_EXAMPLE
# cd /root
# cp -v /etc/boxbackup/bbackupd/boxbackup-client-cert-req.pem ca/clients/1-csr.pem
# bbstored-certs ca sign ca/clients/1-csr.pem

This certificate is for backup account

   1

Ensure this matches the account number you are expecting. The filename is

   ./bbackupd/boxbackup-client-cert-req.pem

which should include this account number, and additionally, you should check
that you received it from the right person.

Signing the wrong certificate compromises the security of your backup system.

Would you like to sign this certificate? (type 'yes' to confirm)
yes
yes
Signature ok
subject=CN = BACKUP-1
Getting CA Private Key


Certificate signed.

Send the files

   ca/clients/1-cert.pem
   ca/roots/serverCA.pem

to the client.

#+END_EXAMPLE

Now we can install the files:

#+BEGIN_SRC bash
 cp -v ca/clients/1-cert.pem /etc/boxbackup/bbackupd/boxbackup-client-cert.pem
 cp -v ca/roots/serverCA.pem /etc/boxbackup/bbackupd/boxbackup-server-ca-cert.pem
#+END_SRC


Let's restart the client:

#+BEGIN_SRC bash
root@stretch:/root# systemctl restart boxbackup-client
root@stretch:/root# systemctl status boxbackup-client
● boxbackup-client.service - Box Backup Client
   Loaded: loaded (/lib/systemd/system/boxbackup-client.service; disabled; vendor preset: enabled)
   Active: active (running) since Sun 2017-06-18 00:01:20 UTC; 3s ago
 Main PID: 2793 (bbackupd)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/boxbackup-client.service
           └─2793 /usr/sbin/bbackupd -F -c /etc/boxbackup/bbackupd.conf

Jun 18 00:01:20 stretch systemd[1]: Started Box Backup Client.
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE: Box Backup Client v0.12~gitcf52058f-1, (c) Ben Summers and contributors 2003-2014
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE: Starting daemon, version: 0.12~gitcf52058f-1
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE:  Starting daemon, version: 0.12~gitcf52058f-1
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE: Using configuration file: /etc/boxbackup/bbackupd.conf
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE:  Using configuration file: /etc/boxbackup/bbackupd.conf
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE: Store object info file is not enabled. Will download directory listings from store.
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE:  Store object info file is not enabled. Will download directory listings from store.
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE: Beginning scan of local files
Jun 18 00:01:20 stretch bbackupd[2793]: NOTICE:  Beginning scan of local files
#+END_SRC

And now let's do a backup. This may take a while...

#+BEGIN_EXAMPLE
bbackupctl sync-and-wait
NOTICE:  Using configuration file /etc/boxbackup/bbackupd.conf
INFO:    Daemon configuration summary:
  AutomaticBackup = false
  UpdateStoreInterval = 0 seconds
  MinimumFileAge = 0 seconds
  MaxUploadWait = 0 seconds
INFO:    Sync started...
INFO:    Sync finished.
#+END_EXAMPLE

Let's check the size of the backup store, and the number of files in backup:

#+BEGIN_EXAMPLE
root@stretch:/tmp# bbackupquery 'list -R'  exit | wc -l
1184
root@stretch:/tmp# du -sh /boxbackup
5.1M	/boxbackup
root@stretch:/tmp# 
#+END_EXAMPLE

For automated installation, here are my boxbackup settings:

#+BEGIN_EXAMPLE
root@stretch:/tmp# debconf-get-selections | grep boxbackup
debconf-get-selections | grep boxbackup
boxbackup-client    boxbackup-client/MaxUploadWait    string    86400
boxbackup-client    boxbackup-client/notifyMail    string    root
boxbackup-client    boxbackup-client/accountNumber    string    1
boxbackup-client    boxbackup-client/UpdateStoreInterval    string    3600
boxbackup-client    boxbackup-client/MinimumFileAge    string    21600
boxbackup-server    boxbackup-server/generateCertificate    boolean    true
boxbackup-server    boxbackup-server/raidBlockSize    string    4096
boxbackup-server    boxbackup-server/debconf    boolean    true
boxbackup-client    boxbackup-client/backupMode    select    snapshot
boxbackup-client    boxbackup-client/backupServer    string    localhost
boxbackup-client    boxbackup-client/backupDirs    string    /etc /home
boxbackup-server    boxbackup-server/raidDirectories    string    /boxbackup/0
boxbackup-client    boxbackup-client/generateCertificate    boolean    true
boxbackup-client    boxbackup-client/debconf    boolean    true
#+END_EXAMPLE