summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorNikolaus Rath <Nikolaus@rath.org>2016-03-09 10:08:24 -0800
committerNikolaus Rath <Nikolaus@rath.org>2016-03-09 10:08:24 -0800
commit538edbe3d8a600a9ee4ff4a7822abbc79cbf00e1 (patch)
tree21cd3a3ec21703fc89ffde47c88f23b2f4a3e785 /doc
parent5b62499deb280253671a3c468644335be9e55f04 (diff)
Import s3ql_1.2.orig.tar.bz2
Diffstat (limited to 'doc')
-rw-r--r--doc/html/.buildinfo2
-rw-r--r--doc/html/_sources/about.txt16
-rw-r--r--doc/html/_sources/adm.txt4
-rw-r--r--doc/html/_sources/backends.txt415
-rw-r--r--doc/html/_sources/contrib.txt6
-rw-r--r--doc/html/_sources/general.txt149
-rw-r--r--doc/html/_sources/index.txt1
-rw-r--r--doc/html/_sources/installation.txt18
-rw-r--r--doc/html/_sources/issues.txt7
-rw-r--r--doc/html/_sources/man/adm.txt13
-rw-r--r--doc/html/_sources/man/cp.txt8
-rw-r--r--doc/html/_sources/man/ctrl.txt6
-rw-r--r--doc/html/_sources/man/fsck.txt15
-rw-r--r--doc/html/_sources/man/lock.txt6
-rw-r--r--doc/html/_sources/man/mkfs.txt18
-rw-r--r--doc/html/_sources/man/mount.txt14
-rw-r--r--doc/html/_sources/man/pcp.txt4
-rw-r--r--doc/html/_sources/man/rm.txt6
-rw-r--r--doc/html/_sources/mkfs.txt9
-rw-r--r--doc/html/_sources/mount.txt32
-rw-r--r--doc/html/_sources/special.txt4
-rw-r--r--doc/html/_sources/tips.txt23
-rw-r--r--doc/html/about.html22
-rw-r--r--doc/html/adm.html48
-rw-r--r--doc/html/backends.html386
-rw-r--r--doc/html/contrib.html27
-rw-r--r--doc/html/fsck.html38
-rw-r--r--doc/html/general.html251
-rw-r--r--doc/html/index.html30
-rw-r--r--doc/html/installation.html51
-rw-r--r--doc/html/issues.html35
-rw-r--r--doc/html/man/adm.html69
-rw-r--r--doc/html/man/cp.html35
-rw-r--r--doc/html/man/ctrl.html17
-rw-r--r--doc/html/man/expire_backups.html13
-rw-r--r--doc/html/man/fsck.html69
-rw-r--r--doc/html/man/index.html11
-rw-r--r--doc/html/man/lock.html15
-rw-r--r--doc/html/man/mkfs.html77
-rw-r--r--doc/html/man/mount.html95
-rw-r--r--doc/html/man/pcp.html14
-rw-r--r--doc/html/man/rm.html15
-rw-r--r--doc/html/man/stat.html11
-rw-r--r--doc/html/man/umount.html11
-rw-r--r--doc/html/mkfs.html50
-rw-r--r--doc/html/mount.html118
-rw-r--r--doc/html/objects.invbin505 -> 521 bytes
-rw-r--r--doc/html/resources.html11
-rw-r--r--doc/html/search.html11
-rw-r--r--doc/html/searchindex.js2
-rw-r--r--doc/html/special.html63
-rw-r--r--doc/html/tips.html34
-rw-r--r--doc/html/umount.html19
-rw-r--r--doc/latex/manual.aux589
-rw-r--r--doc/latex/manual.log173
-rw-r--r--doc/latex/manual.out107
-rw-r--r--doc/latex/manual.tex921
-rw-r--r--doc/latex/manual.toc258
-rw-r--r--doc/man/fsck.s3ql.164
-rw-r--r--doc/man/mkfs.s3ql.173
-rw-r--r--doc/man/mount.s3ql.184
-rw-r--r--doc/man/s3qladm.163
-rw-r--r--doc/man/s3qlcp.117
-rw-r--r--doc/man/s3qlctrl.114
-rw-r--r--doc/man/s3qllock.110
-rw-r--r--doc/man/s3qlrm.110
-rw-r--r--doc/man/s3qlstat.15
-rw-r--r--doc/man/umount.s3ql.16
-rw-r--r--doc/manual.pdfbin289155 -> 286628 bytes
69 files changed, 2377 insertions, 2441 deletions
diff --git a/doc/html/.buildinfo b/doc/html/.buildinfo
index 7d05252..19a9de7 100644
--- a/doc/html/.buildinfo
+++ b/doc/html/.buildinfo
@@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 85f6a556ea11bab9efd893c7b6e319a1
+config: 511abf8af0910e88e7d158947974ebc2
tags: fbb0d17656682115ca4d033fb2f83ba1
diff --git a/doc/html/_sources/about.txt b/doc/html/_sources/about.txt
index bef3684..d3fc8a5 100644
--- a/doc/html/_sources/about.txt
+++ b/doc/html/_sources/about.txt
@@ -4,13 +4,15 @@
About S3QL
============
-S3QL is a file system that stores all its data online. It supports
-`Amazon S3 <http://aws.amazon.com/s3 Amazon S3>`_ as well as arbitrary
-SFTP servers and effectively provides you with a hard disk of dynamic,
-infinite capacity that can be accessed from any computer with internet
-access.
-
-S3QL is providing a standard, full featured UNIX file system that is
+S3QL is a file system that stores all its data online using storage
+services like `Google Storage
+<http://code.google.com/apis/storage/>`_, `Amazon S3
+<http://aws.amazon.com/s3 Amazon S3>`_ or `OpenStack
+<http://openstack.org/projects/storage/>`_. S3QL effectively provides
+a hard disk of dynamic, infinite capacity that can be accessed from
+any computer with internet access.
+
+S3QL is a standard conforming, full featured UNIX file system that is
conceptually indistinguishable from any local file system.
Furthermore, S3QL has additional features like compression,
encryption, data de-duplication, immutable trees and snapshotting
diff --git a/doc/html/_sources/adm.txt b/doc/html/_sources/adm.txt
index 3e50f64..34fc7b4 100644
--- a/doc/html/_sources/adm.txt
+++ b/doc/html/_sources/adm.txt
@@ -14,7 +14,7 @@ The syntax is ::
s3qladm [options] <action> <storage-url>
where :var:`action` may be either of :program:`passphrase`,
-:program:`upgrade`, :program:`delete` or :program:`download-metadata`.
+:program:`upgrade`, :program:`clear` or :program:`download-metadata`.
The :program:`s3qladm` accepts the following general options, no
matter what specific action is being invoked:
@@ -55,7 +55,7 @@ Deleting a file system
A file system can be deleted with::
- s3qladm delete <storage url>
+ s3qladm clear <storage url>
This physically deletes all the data and file system structures.
diff --git a/doc/html/_sources/backends.txt b/doc/html/_sources/backends.txt
index 480ff90..5bc9521 100644
--- a/doc/html/_sources/backends.txt
+++ b/doc/html/_sources/backends.txt
@@ -1,292 +1,205 @@
.. -*- mode: rst -*-
+.. _storage_backends:
+
==================
Storage Backends
==================
-S3QL can use different protocols to store the file system data.
-Independent of the backend that you use, the place where your file
-system data is being stored is called a *bucket*. (This is mostly for
-historical reasons, since initially S3QL supported only the Amazon S3
-backend).
-
-
-On Backend Reliability
-======================
-
-S3QL has been designed for use with a storage backend where data loss
-is so infrequent that it can be completely neglected (e.g. the Amazon
-S3 backend). If you decide to use a less reliable backend, you should
-keep the following warning in mind and read this section carefully.
-
-.. WARNING::
-
- S3QL is not able to compensate for any failures of the backend. In
- particular, it is not able reconstruct any data that has been lost
- or corrupted by the backend. The persistence and durability of data
- stored in an S3QL file system is limited and determined by the
- backend alone.
-
-
-On the plus side, if a backend looses or corrupts some of the stored
-data, S3QL *will* detect the problem. Missing data will be detected
-when running `fsck.s3ql` or when attempting to access the data in the
-mounted file system. In the later case you will get an IO Error, and
-on unmounting S3QL will warn you that the file system is damaged and
-you need to run `fsck.s3ql`.
-
-`fsck.s3ql` will report all the affected files and move them into the
-`/lost+found` directory of the file system.
-
-You should be aware that, because of S3QL's data de-duplication
-feature, the consequences of a data loss in the backend can be
-significantly more severe than you may expect. More concretely, a data
-loss in the backend at time *x* may cause data that is written *after*
-time *x* to be lost as well. What may happen is this:
-
-#. You store an important file in the S3QL file system.
-#. The backend looses the data blocks of this file. As long as you
- do not access the file or run `fsck.s3ql`, S3QL
- is not aware that the data has been lost by the backend.
-#. You save an additional copy of the important file in a different
- location on the same S3QL file system.
-#. S3QL detects that the contents of the new file are identical to the
- data blocks that have been stored earlier. Since at this point S3QL
- is not aware that these blocks have been lost by the backend, it
- does not save another copy of the file contents in the backend but
- relies on the (presumably) existing blocks instead.
-#. Therefore, even though you saved another copy, you still do not
- have a backup of the important file (since both copies refer to the
- same data blocks that have been lost by the backend).
-
-As one can see, this effect becomes the less important the more often
-one runs `fsck.s3ql`, since `fsck.s3ql` will make S3QL aware of any
-blocks that the backend may have lost. Figuratively, this establishes
-a "checkpoint": data loss in the backend that occurred before running
-`fsck.s3ql` can not affect any file system operations performed after
-running `fsck.s3ql`.
-
-
-Nevertheless, (as said at the beginning) the recommended way to use
-S3QL is in combination with a sufficiently reliable storage backend.
-In that case none of the above will ever be a concern.
-
-
-The `authinfo` file
-===================
-
-Most backends first try to read the file `~/.s3ql/authinfo` to determine
-the username and password for connecting to the remote host. If this
-fails, both username and password are read from the terminal.
-
-The `authinfo` file has to contain entries of the form ::
-
- backend <backend> machine <host> login <user> password <password>
-
-So to use the login `joe` with password `jibbadup` when using the FTP
-backend to connect to the host `backups.joesdomain.com`, you would
-specify ::
-
- backend ftp machine backups.joesdomain.com login joe password jibbadup
+The following backends are currently available in S3QL:
+
+Google Storage
+==============
+
+`Google Storage <http://code.google.com/apis/storage/>`_ is an online
+storage service offered by Google. It is the most feature-rich service
+supported by S3QL and S3QL offers the best performance when used with
+the Google Storage backend.
+
+To use the Google Storage backend, you need to have (or sign up for) a
+Google account, and then `activate Google Storage
+<http://code.google.com/apis/storage/docs/signup.html>`_ for your
+account. The account is free, you will pay only for the amount of
+storage and traffic that you actually use. Once you have created the
+account, make sure to `activate legacy access
+<http://code.google.com/apis/storage/docs/reference/v1/apiversion1.html#enabling>`_.
+
+To create a Google Storage bucket, you can use e.g. the `Google
+Storage Manager
+<https://sandbox.google.com/storage/>`_. The
+storage URL for accessing the bucket in S3QL is then ::
+
+ gs://<bucketname>/<prefix>
+
+Here *bucketname* is the name of the bucket, and *prefix* can be
+an arbitrary prefix that will be prepended to all object names used by
+S3QL. This allows you to store several S3QL file systems in the same
+Google Storage bucket.
+
+Note that the backend login and password for accessing your Google
+Storage bucket are not your Google account name and password, but the
+*Google Storage developer access key* and *Google Storage developer
+secret* that you can manage with the `Google Storage key management
+tool
+<https://code.google.com/apis/console/#:storage:legacy>`_.
+
+If you would like S3QL to connect using HTTPS instead of standard
+HTTP, start the storage url with ``gss://`` instead of ``gs://``. Note
+that at this point S3QL does not perform any server certificate
+validation (see `issue 267
+<http://code.google.com/p/s3ql/issues/detail?id=267>`_).
+
+
+Amazon S3
+=========
+
+`Amazon S3 <http://aws.amazon.com/s3>`_ is the online storage service
+offered by `Amazon Web Services (AWS) <http://aws.amazon.com/>`_. To
+use the S3 backend, you first need to sign up for an AWS account. The
+account is free, you will pay only for the amount of storage and
+traffic that you actually use. After that, you need to create a bucket
+that will hold the S3QL file system, e.g. using the `AWS Management
+Console <https://console.aws.amazon.com/s3/home>`_. For best
+performance, it is recommend to create the bucket in the
+geographically closest storage region, but not the US Standard
+region (see below).
+
+The storage URL for accessing S3 buckets in S3QL has the form ::
+
+ s3://<bucketname>/<prefix>
+
+Here *bucketname* is the name of the bucket, and *prefix* can be
+an arbitrary prefix that will be prepended to all object names used by
+S3QL. This allows you to store several S3QL file systems in the same
+S3 bucket.
+
+Note that the backend login and password for accessing S3 are not the
+user id and password that you use to log into the Amazon Webpage, but
+the *AWS access key id* and *AWS secret access key* shown under `My
+Account/Access Identifiers
+<https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key>`_.
-
-Consistency Guarantees
-======================
-
-The different backends provide different types of *consistency
-guarantees*. Informally, a consistency guarantee tells you how fast
-the backend will apply changes to the stored data.
+If you would like S3QL to connect using HTTPS instead of standard
+HTTP, start the storage url with ``s3s://`` instead of ``s3://``. Note
+that, as of May 2011, Amazon S3 is faster when accessed using a
+standard HTTP connection, and that S3QL does not perform any server
+certificate validation (see `issue 267
+<http://code.google.com/p/s3ql/issues/detail?id=267>`_).
-S3QL defines the following three levels:
-* **Read-after-Write Consistency.** This is the strongest consistency
- guarantee. If a backend offers read-after-write consistency, it
- guarantees that as soon as you have committed any changes to the
- backend, subsequent requests will take into account these changes.
+Reduced Redundancy Storage (RRS)
+--------------------------------
-* **Read-after-Create Consistency.** If a backend provides only
- read-after-create consistency, only the creation of a new object is
- guaranteed to be taken into account for subsequent requests. This
- means that, for example, if you overwrite data in an existing
- object, subsequent requests may still return the old data for a
- certain period of time.
+S3QL does not allow the use of `reduced redundancy storage
+<http://aws.amazon.com/s3/#protecting>`_. The reason for that is a
+combination of three factors:
-* **Eventual consistency.** This is the lowest consistency level.
- Basically, any changes that you make to the backend may not be
- visible for a certain amount of time after the change has been made.
- However, you are guaranteed that no change will be lost. All changes
- will *eventually* become visible.
-
- .
+* RRS has a relatively low reliability, on average you loose one
+ out of every ten-thousand objects a year. So you can expect to
+ occasionally loose some data.
+* When `fsck.s3ql` asks S3 for a list of the stored objects, this list
+ includes even those objects that have been lost. Therefore
+ `fsck.s3ql` *can not detect lost objects* and lost data will only
+ become apparent when you try to actually read from a file whose data
+ has been lost. This is a (very unfortunate) peculiarity of Amazon
+ S3.
-As long as your backend provides read-after-write or read-after-create
-consistency, you do not have to worry about consistency guarantees at
-all. However, if you plan to use a backend with only eventual
-consistency, you have to be a bit careful in some situations.
-
-
-.. _eventual_consistency:
-
-Dealing with Eventual Consistency
----------------------------------
+* Due to the data de-duplication feature of S3QL, unnoticed lost
+ objects may cause subsequent data loss later in time (see
+ :ref:`backend_reliability` for details).
-.. NOTE::
- The following applies only to storage backends that do not provide
- read-after-create or read-after-write consistency. Currently,
- this is only the Amazon S3 backend *if used with the US-Standard
- storage region*. If you use a different storage backend, or the S3
- backend with a different storage region, this section does not apply
- to you.
+Potential issues when using the US Standard storage region
+----------------------------------------------------------
-While the file system is mounted, S3QL is able to automatically handle
-all issues related to the weak eventual consistency guarantee.
-However, some issues may arise during the mount process and when the
-file system is checked.
+In the US Standard storage region, Amazon S3 does not guarantee read
+after create consistency. This means that after a new object has been
+stored, requests to read this object may still fail for a little
+while. While the file system is mounted, S3QL is able to automatically
+handle all issues related to this so-called eventual consistency.
+However, problems may arise during the mount process and when the file
+system is checked:
Suppose that you mount the file system, store some new data, delete
-some old data and unmount it again. Now remember that eventual
-consistency means that there is no guarantee that these changes will
-be visible immediately. At least in theory it is therefore possible
-that if you mount the file system again, S3QL does not see any of the
-changes that you have done and presents you an "old version" of the
-file system without them. Even worse, if you notice the problem and
-unmount the file system, S3QL will upload the old status (which S3QL
-necessarily has to consider as current) and thereby permanently
-override the newer version (even though this change may not become
-immediately visible either).
-
-The same problem applies when checking the file system. If the backend
+some old data and unmount it again. Now there is no guarantee that
+these changes will be visible immediately. At least in theory it is
+therefore possible that if you mount the file system again, S3QL
+does not see any of the changes that you have done and presents you
+an "old version" of the file system without them. Even worse, if you
+notice the problem and unmount the file system, S3QL will upload the
+old status (which S3QL necessarily has to consider as current) and
+thereby permanently override the newer version (even though this
+change may not become immediately visible either).
+
+The same problem applies when checking the file system. If S3
provides S3QL with only partially updated data, S3QL has no way to
find out if this a real consistency problem that needs to be fixed or
if it is only a temporary problem that will resolve itself
automatically (because there are still changes that have not become
visible yet).
-While this may seem to be a rather big problem, the likelihood of it
-to occur is rather low. In practice, most storage providers rarely
-need more than a few seconds to apply incoming changes, so to trigger
-this problem one would have to unmount and remount the file system in
-a very short time window. Many people therefore make sure that they
-wait a few minutes between successive mounts (or file system checks)
-and decide that the remaining risk is negligible.
-
-Nevertheless, the eventual consistency guarantee does not impose an
-upper limit on the time that it may take for change to become visible.
-Therefore there is no "totally safe" waiting time that would totally
-eliminate this problem; a theoretical possibility always remains.
-
-
-
-The Amazon S3 Backend
-=====================
-
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form `s3://<bucketname>`. Bucket names must conform to the `S3
-Bucket Name Restrictions`_.
-
-The S3 backend offers exceptionally strong reliability guarantees. As
-of August 2010, Amazon guarantees a durability of 99.999999999% per
-year. In other words, if you store a thousand million objects then on
-average you would loose less than one object in a hundred years.
-
-The Amazon S3 backend provides read-after-create consistency for the
-EU, Asia-Pacific and US-West storage regions. *For the US-Standard
-storage region, Amazon S3 provides only eventual consistency* (please
-refer to :ref:`eventual_consistency` for information about
-what this entails).
-
-When connecting to Amazon S3, S3QL uses an unencrypted HTTP
-connection, so if you want your data to stay confidential, you have
-to create the S3QL file system with encryption (this is also the default).
-
-When reading the authentication information for the S3 backend from
-the `authinfo` file, the `host` field is ignored, i.e. the first entry
-with `s3` as a backend will be used. For example ::
-
- backend s3 machine any login myAWSaccessKeyId password myAwsSecretAccessKey
-
-Note that the bucket names come from a global pool, so chances are
-that your favorite name has already been taken by another S3 user.
-Usually a longer bucket name containing some random numbers, like
-`19283712_yourname_s3ql`, will work better.
-
-If you do not already have one, you need to obtain an Amazon S3
-account from `Amazon AWS <http://aws.amazon.com/>`_. The account is
-free, you will pay only for the amount of storage that you actually
-use.
-
-Note that the login and password for accessing S3 are not the user id
-and password that you use to log into the Amazon Webpage, but the "AWS
-access key id" and "AWS secret access key" shown under `My
-Account/Access Identifiers
-<https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key>`_.
-
-.. _`S3 Bucket Name Restrictions`: http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/BucketRestrictions.html
-
-.. NOTE::
-
- S3QL also allows you to use `reduced redundancy storage
- <http://aws.amazon.com/s3/#protecting>`_ by using ``s3rr://``
- instead of ``s3://`` in the storage url. However, this not
- recommended. The reason is a combination of three factors:
-
- * RRS has a relatively low reliability, on average you loose one
- out of every ten-thousand objects a year. So you can expect to
- occasionally loose some data.
-
- * When `fsck.s3ql` asks Amazon S3 for a list of the stored objects,
- this list includes even those objects that have been lost.
- Therefore `fsck.s3ql` *can not detect lost objects* and lost data
- will only become apparent when you try to actually read from a
- file whose data has been lost. This is a (very unfortunate)
- peculiarity of Amazon S3.
+The likelihood of this to happen is rather low. In practice, most
+objects are ready for retrieval just a few seconds after they have
+been stored, so to trigger this problem one would have to unmount and
+remount the file system in a very short time window. However, since S3
+does not place any upper limit on the length of this window, it is
+recommended to not place S3QL buckets in the US Standard storage
+region. As of May 2011, all other storage regions provide stronger
+consistency guarantees that completely eliminate any of the described
+problems.
- * Due to the data de-duplication feature of S3QL, unnoticed lost
- objects may cause subsequent data loss later in time (see `On
- Backend Reliability`_ for details).
- In other words, you should really only store an S3QL file system
- using RRS if you know exactly what you are getting into.
-
+S3 compatible
+=============
+S3QL is also able to access other, S3 compatible storage services for
+which no specific backend exists. Note that when accessing such
+services, only the lowest common denominator of available features can
+be used, so it is generally recommended to use a service specific
+backend instead.
-The Local Backend
-=================
+The storage URL for accessing an arbitrary S3 compatible storage
+service is ::
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-`local://<path>`. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. `local:///var/archive`.
+ s3c://<hostname>:<port>/<bucketname>/<prefix>
-The local backend provides read-after-write consistency.
+or ::
-The SFTP Backend
-================
+ s3cs://<hostname>:<port>/<bucketname>/<prefix>
-The SFTP backend uses the SFTP protocol, which is a file transfer
-protocol similar to ftp, but uses an encrypted SSH connection.
-It provides read-after-write consistency.
+to use HTTPS connections. Note, however, that at this point S3QL does
+not verify the server certificate (cf. `issue 267
+<http://code.google.com/p/s3ql/issues/detail?id=267>`_).
-Note that the SFTP backend is rather slow and has not been tested
-as extensively as the S3 and Local backends.
-The storage URL for SFTP connections has the form ::
+Local
+=====
- sftp://<host>[:port]/<path>
+S3QL is also able to store its data on the local file system. This can
+be used to backup data on external media, or to access external
+services that S3QL can not talk to directly (e.g., it is possible to
+store data over SSH by first mounting the remote system using
+`sshfs`_, then using the local backend to store the data in the sshfs
+mountpoint).
-The SFTP backend will always ask you for a password if you haven't
-defined one in `~/.s3ql/authinfo`. However, public key authentication
-is tried first and the password will only be used if the public key
-authentication fails.
+The storage URL for local storage is ::
-The public and private keys will be read from the standard files in
-`~/.ssh/`. Note that S3QL will refuse to connect to a computer with
-unknown host key; to add the key to your local keyring you have to
-establish a connection to that computer with the standard SSH command
-line programs first.
+ local://<path>
+
+Note that you have to write three consecutive slashes to specify an
+absolute path, e.g. `local:///var/archive`. Also, relative paths will
+automatically be converted to absolute paths before the authentication
+file is read, i.e. if you are in the `/home/john` directory and try to
+mount `local://bucket`, the corresponding section in the
+authentication file must match the storage url
+`local:///home/john/bucket`.
+SSH/SFTP
+========
+Previous versions of S3QL included an SSH/SFTP backend. With newer
+S3QL versions, it is recommended to instead combine the local backend
+with `sshfs <http://fuse.sourceforge.net/sshfs.html>`_ (cf. :ref:`ssh_tipp`).
diff --git a/doc/html/_sources/contrib.txt b/doc/html/_sources/contrib.txt
index 3ee2323..7cbe997 100644
--- a/doc/html/_sources/contrib.txt
+++ b/doc/html/_sources/contrib.txt
@@ -14,8 +14,10 @@ the `contrib` directory of the source distribution or in
benchmark.py
============
-This program measures your uplink bandwidth and compression speed and
-recommends a compression algorithm for optimal throughput.
+This program measures S3QL write performance, uplink bandwidth and
+compression speed to determine the limiting factor. It also gives
+recommendation for compression algorithm and number of upload threads
+to achieve maximum performance.
s3_copy.py
diff --git a/doc/html/_sources/general.txt b/doc/html/_sources/general.txt
new file mode 100644
index 0000000..e95ab8e
--- /dev/null
+++ b/doc/html/_sources/general.txt
@@ -0,0 +1,149 @@
+.. -*- mode: rst -*-
+
+===================
+General Information
+===================
+
+Terminology
+===========
+
+S3QL can store data at different service providers and using different
+protocols. The term *backend* refers to both the part of S3QL that
+implements communication with a specific storage service and the
+storage service itself. Most backends can hold more than one S3QL file
+system and thus require some additional information that specifies the
+file system location within the backend. This location is called a
+*bucket* (for historical reasons).
+
+Many S3QL commands expect a *storage url* as a parameter. A storage
+url specifies both the backend and the bucket and thus uniquely
+identifies an S3QL file system. The form of the storage url depends on
+the backend and is described together with the
+:ref:`storage_backends`.
+
+.. _bucket_pw:
+
+Storing Authentication Information
+==================================
+
+Normally, S3QL reads username and password for the backend as well as
+an encryption passphrase for the bucket from the terminal. Most
+commands also accept an :cmdopt:`--authfile` parameter that can be
+used to read this information from a file instead.
+
+The authentication file consists of sections, led by a ``[section]``
+header and followed by ``name: value`` entries. The section headers
+themselves are not used by S3QL but have to be unique within the file.
+
+In each section, the following entries can be defined:
+
+:storage-url:
+ Specifies the storage url to which this section applies. If a
+ storage url starts with the value of this entry, the section is
+ considered applicable.
+
+:backend-login:
+ Specifies the username to use for authentication with the backend.
+
+:backend-password:
+ Specifies the password to use for authentication with the backend.
+
+:bucket-passphrase:
+ Specifies the passphrase to use to decrypt the bucket (if it is
+ encrypted).
+
+
+When reading the authentication file, S3QL considers every applicable
+section in order and uses the last value that it found for each entry.
+For example, consider the following authentication file::
+
+ [s3]
+ storage-url: s3://
+ backend-login: joe
+ backend-password: notquitesecret
+
+ [bucket1]
+ storage-url: s3://joes-first-bucket
+ bucket-passphrase: neitheristhis
+
+ [bucket2]
+ storage-url: s3://joes-second-bucket
+ bucket-passphrase: swordfish
+
+ [bucket3]
+ storage-url: s3://joes-second-bucket/with-prefix
+ backend-login: bill
+ backend-password: bi23ll
+ bucket-passphrase: ll23bi
+
+With this authentication file, S3QL would try to log in as "joe"
+whenever the s3 backend is used, except when accessing a storage url
+that begins with "s3://joes-second-bucket/with-prefix". In that case,
+the last section becomes active and S3QL would use the "bill"
+credentials. Furthermore, bucket encryption passphrases will be used
+for storage urls that start with "s3://joes-first-bucket" or
+"s3://joes-second-bucket".
+
+The authentication file is parsed by the `Python ConfigParser
+module <http://docs.python.org/library/configparser.html>`_.
+
+.. _backend_reliability:
+
+On Backend Reliability
+======================
+
+S3QL has been designed for use with a storage backend where data loss
+is so infrequent that it can be completely neglected (e.g. the Amazon
+S3 backend). If you decide to use a less reliable backend, you should
+keep the following warning in mind and read this section carefully.
+
+.. WARNING::
+
+ S3QL is not able to compensate for any failures of the backend. In
+ particular, it is not able reconstruct any data that has been lost
+ or corrupted by the backend. The persistence and durability of data
+ stored in an S3QL file system is limited and determined by the
+ backend alone.
+
+
+On the plus side, if a backend looses or corrupts some of the stored
+data, S3QL *will* detect the problem. Missing data will be detected
+when running `fsck.s3ql` or when attempting to access the data in the
+mounted file system. In the later case you will get an IO Error, and
+on unmounting S3QL will warn you that the file system is damaged and
+you need to run `fsck.s3ql`.
+
+`fsck.s3ql` will report all the affected files and move them into the
+`/lost+found` directory of the file system.
+
+You should be aware that, because of S3QL's data de-duplication
+feature, the consequences of a data loss in the backend can be
+significantly more severe than you may expect. More concretely, a data
+loss in the backend at time *x* may cause data that is written *after*
+time *x* to be lost as well. What may happen is this:
+
+#. You store an important file in the S3QL file system.
+#. The backend looses the data blocks of this file. As long as you
+ do not access the file or run `fsck.s3ql`, S3QL
+ is not aware that the data has been lost by the backend.
+#. You save an additional copy of the important file in a different
+ location on the same S3QL file system.
+#. S3QL detects that the contents of the new file are identical to the
+ data blocks that have been stored earlier. Since at this point S3QL
+ is not aware that these blocks have been lost by the backend, it
+ does not save another copy of the file contents in the backend but
+ relies on the (presumably) existing blocks instead.
+#. Therefore, even though you saved another copy, you still do not
+ have a backup of the important file (since both copies refer to the
+ same data blocks that have been lost by the backend).
+
+As one can see, this effect becomes the less important the more often
+one runs `fsck.s3ql`, since `fsck.s3ql` will make S3QL aware of any
+blocks that the backend may have lost. Figuratively, this establishes
+a "checkpoint": data loss in the backend that occurred before running
+`fsck.s3ql` can not affect any file system operations performed after
+running `fsck.s3ql`.
+
+Nevertheless, the recommended way to use S3QL is in combination with a
+sufficiently reliable storage backend. In that case none of the above
+will ever be a concern.
diff --git a/doc/html/_sources/index.txt b/doc/html/_sources/index.txt
index f3b5b72..960a63a 100644
--- a/doc/html/_sources/index.txt
+++ b/doc/html/_sources/index.txt
@@ -9,6 +9,7 @@
about
installation
+ general
backends
mkfs
adm
diff --git a/doc/html/_sources/installation.txt b/doc/html/_sources/installation.txt
index b57325e..e153de0 100644
--- a/doc/html/_sources/installation.txt
+++ b/doc/html/_sources/installation.txt
@@ -27,13 +27,11 @@ running S3QL. Generally, you should first check if your distribution
already provides a suitable packages and only install from source if
that is not the case.
-* Kernel version 2.6.9 or newer. Starting with kernel 2.6.26
- you will get significantly better write performance, so you should
- actually use *2.6.26 or newer whenever possible*.
-
-* The `FUSE Library <http://fuse.sourceforge.net/>`_ should already be
- installed on your system. However, you have to make sure that you
- have at least version 2.8.0.
+* Kernel: Linux 2.6.9 or newer or FreeBSD with `FUSE4BSD
+ <http://www.freshports.org/sysutils/fusefs-kmod/>`_. Starting with
+ kernel 2.6.26 you will get significantly better write performance,
+ so under Linux you should actually use *2.6.26 or newer whenever
+ possible*.
* The `PyCrypto++ Python Module
<http://pypi.python.org/pypi/pycryptopp>`_. To check if this module
@@ -69,12 +67,6 @@ that is not the case.
this module. If you are upgrading from such a version, make sure to
completely remove the old S3QL version first.
-* If you want to use the SFTP backend, then you also need the
- `Paramiko Python Module <http://www.lag.net/paramiko/>`_. To check
- if this module is installed, try to execute `python -c 'import
- paramiko'`.
-
-
.. _inst-s3ql:
Installing S3QL
diff --git a/doc/html/_sources/issues.txt b/doc/html/_sources/issues.txt
index 29b76ce..ac2cb8c 100644
--- a/doc/html/_sources/issues.txt
+++ b/doc/html/_sources/issues.txt
@@ -4,6 +4,11 @@
Known Issues
============
+* S3QL does not verify TLS/SSL server certificates, so a
+ man-in-the-middle attack is principally possible. See `issue 267
+ <http://code.google.com/p/s3ql/issues/detail?id=267>`_ for more
+ details.
+
* S3QL is rather slow when an application tries to write data in
unreasonably small chunks. If a 1 MB file is copied in chunks of 1
KB, this will take more than 10 times as long as when it's copied
@@ -74,7 +79,7 @@ Known Issues
* S3QL relies on the backends not to run out of space. This is a given
for big storage providers like Amazon S3, but you may stumble upon
- this if you store buckets e.g. on a small sftp server.
+ this if you store buckets e.g. on smaller servers or servies.
If there is no space left in the backend, attempts to write more
data into the S3QL file system will fail and the file system will be
diff --git a/doc/html/_sources/man/adm.txt b/doc/html/_sources/man/adm.txt
index c23865e..9010cd8 100644
--- a/doc/html/_sources/man/adm.txt
+++ b/doc/html/_sources/man/adm.txt
@@ -23,8 +23,8 @@ The |command| command performs various operations on S3QL buckets.
The file system contained in the bucket *must not be mounted* when
using |command| or things will go wrong badly.
-.. include:: ../include/backends.rst
-
+The storage url depends on the backend that is used. The S3QL User's
+Guide should be consulted for a description of the available backends.
Options
=======
@@ -52,15 +52,6 @@ download-metadata
Interactively download backups of the file system metadata.
-Files
-=====
-
-Authentication data for backends and bucket encryption passphrases are
-read from :file:`authinfo` in :file:`~/.s3ql` or the directory
-specified with :cmdopt:`--homedir`. Log files are placed in the same
-directory.
-
-
.. include:: ../include/postman.rst
.. |command| replace:: :program:`s3qladm`
diff --git a/doc/html/_sources/man/cp.txt b/doc/html/_sources/man/cp.txt
index d0cbb41..a397fe9 100644
--- a/doc/html/_sources/man/cp.txt
+++ b/doc/html/_sources/man/cp.txt
@@ -40,7 +40,7 @@ Note that:
usage of `s3qlcp` is to regularly duplicate the same source
directory, say `documents`, to different target directories. For a
e.g. monthly replication, the target directories would typically be
- named something like `documents_Januray` for the replication in
+ named something like `documents_January` for the replication in
January, `documents_February` for the replication in February etc.
In this case it is clear that the target directories should be
regarded as snapshots of the source directory.
@@ -51,12 +51,6 @@ Note that:
completely (so that S3QL had to fetch all the data over the network
from the backend) before writing them into the destination folder.
-* Before starting with the replication, S3QL has to flush the local
- cache. So if you just copied lots of new data into the file system
- that has not yet been uploaded, replication will take longer than
- usual.
-
-
Snapshotting vs Hardlinking
---------------------------
diff --git a/doc/html/_sources/man/ctrl.txt b/doc/html/_sources/man/ctrl.txt
index 4afa33b..8173162 100644
--- a/doc/html/_sources/man/ctrl.txt
+++ b/doc/html/_sources/man/ctrl.txt
@@ -24,6 +24,12 @@ Description
The |command| command performs various actions on the S3QL file system mounted
in :var:`mountpoint`.
+|command| can only be called by the user that mounted the file system
+and (if the file system was mounted with :cmdopt:`--allow-other` or
+:cmdopt:`--allow-root`) the root user. This limitation might be
+removed in the future (see `issue 155
+<http://code.google.com/p/s3ql/issues/detail?id=155>`_).
+
The following actions may be specified:
flushcache
diff --git a/doc/html/_sources/man/fsck.txt b/doc/html/_sources/man/fsck.txt
index ef6ed2d..8f901c7 100644
--- a/doc/html/_sources/man/fsck.txt
+++ b/doc/html/_sources/man/fsck.txt
@@ -18,10 +18,9 @@ Description
The |command| command checks the new file system in the location
specified by *storage url* for errors and attempts to repair any
-problems.
-
-.. include:: ../include/backends.rst
-
+problems. The storage url depends on the backend that is used. The
+S3QL User's Guide should be consulted for a description of the
+available backends.
Options
=======
@@ -31,14 +30,6 @@ The |command| command accepts the following options.
.. pipeinclude:: ../../bin/fsck.s3ql --help
:start-after: show this help message and exit
-Files
-=====
-
-Authentication data for backends and bucket encryption passphrases are
-read from :file:`authinfo` in :file:`~/.s3ql` or the directory
-specified with :cmdopt:`--homedir`. Log files are placed in the same
-directory.
-
.. include:: ../include/postman.rst
.. |command| replace:: :command:`mkfs.s3ql`
diff --git a/doc/html/_sources/man/lock.txt b/doc/html/_sources/man/lock.txt
index f17bf32..19c3e3a 100644
--- a/doc/html/_sources/man/lock.txt
+++ b/doc/html/_sources/man/lock.txt
@@ -23,6 +23,12 @@ whatsoever. You can not add new files or directories and you can not
change or delete existing files and directories. The only way to get
rid of an immutable tree is to use the :program:`s3qlrm` command.
+|command| can only be called by the user that mounted the file system
+and (if the file system was mounted with :cmdopt:`--allow-other` or
+:cmdopt:`--allow-root`) the root user. This limitation might be
+removed in the future (see `issue 155
+<http://code.google.com/p/s3ql/issues/detail?id=155>`_).
+
Rationale
=========
diff --git a/doc/html/_sources/man/mkfs.txt b/doc/html/_sources/man/mkfs.txt
index c61270a..3a61717 100644
--- a/doc/html/_sources/man/mkfs.txt
+++ b/doc/html/_sources/man/mkfs.txt
@@ -17,10 +17,15 @@ Description
.. include:: ../include/about.rst
The |command| command creates a new file system in the location
-specified by *storage url*.
+specified by *storage url*. The storage url depends on the backend
+that is used. The S3QL User's Guide should be consulted for a
+description of the available backends.
+
+Unless you have specified the `--plain` option, `mkfs.s3ql` will ask
+you to enter an encryption password. This password will *not* be read
+from an authentication file specified with the :cmdopt:`--authfile`
+option to prevent accidental creation of an encrypted bucket.
-.. include:: ../include/backends.rst
-
Options
=======
@@ -30,13 +35,6 @@ The |command| command accepts the following options.
.. pipeinclude:: ../../bin/mkfs.s3ql --help
:start-after: show this help message and exit
-Files
-=====
-
-Authentication data for backends and bucket encryption passphrases are
-read from :file:`authinfo` in :file:`~/.s3ql` or the directory
-specified with :cmdopt:`--homedir`. Log files are placed in the same
-directory.
.. include:: ../include/postman.rst
diff --git a/doc/html/_sources/man/mount.txt b/doc/html/_sources/man/mount.txt
index 3905c03..fb10a11 100644
--- a/doc/html/_sources/man/mount.txt
+++ b/doc/html/_sources/man/mount.txt
@@ -19,10 +19,10 @@ Description
.. include:: ../include/about.rst
The |command| command mounts the S3QL file system stored in *storage
-url* in the directory *mount point*.
+url* in the directory *mount point*. The storage url depends on the
+backend that is used. The S3QL User's Guide should be consulted for a
+description of the available backends.
-.. include:: ../include/backends.rst
-
Options
=======
@@ -33,14 +33,6 @@ The |command| command accepts the following options.
:start-after: show this help message and exit
-Files
-=====
-
-Authentication data for backends and bucket encryption passphrases are
-read from :file:`authinfo` in :file:`~/.s3ql` or the directory
-specified with :cmdopt:`--homedir`. Log files are placed in the same
-directory.
-
.. include:: ../include/postman.rst
diff --git a/doc/html/_sources/man/pcp.txt b/doc/html/_sources/man/pcp.txt
index cd7a66c..c7b3bef 100644
--- a/doc/html/_sources/man/pcp.txt
+++ b/doc/html/_sources/man/pcp.txt
@@ -21,6 +21,10 @@ The |command| command is a is a wrapper that starts several
allows much better copying performance on file system that have
relatively high latency when retrieving individual files like S3QL.
+**Note**: Using this program only improves performance when copying
+*from* an S3QL file system. When copying *to* an S3QL file system,
+using |command| is more likely to *decrease* performance.
+
Options
=======
diff --git a/doc/html/_sources/man/rm.txt b/doc/html/_sources/man/rm.txt
index 0832e27..dda1eda 100644
--- a/doc/html/_sources/man/rm.txt
+++ b/doc/html/_sources/man/rm.txt
@@ -25,6 +25,12 @@ you to delete immutable trees (which can be created with
Be warned that there is no additional confirmation. The directory will
be removed entirely and immediately.
+
+|command| can only be called by the user that mounted the file system
+and (if the file system was mounted with :cmdopt:`--allow-other` or
+:cmdopt:`--allow-root`) the root user. This limitation might be
+removed in the future (see `issue 155
+<http://code.google.com/p/s3ql/issues/detail?id=155>`_).
Options
diff --git a/doc/html/_sources/mkfs.txt b/doc/html/_sources/mkfs.txt
index 0b9fa97..3eceb0a 100644
--- a/doc/html/_sources/mkfs.txt
+++ b/doc/html/_sources/mkfs.txt
@@ -14,7 +14,8 @@ This command accepts the following options:
.. pipeinclude:: ../bin/mkfs.s3ql --help
:start-after: show this help message and exit
-Unless you have specified the `--plain` option, `mkfs.s3ql` will ask you
-to enter an encryption password. If you do not want to enter this
-password every time that you mount the file system, you can store it
-in the `~/.s3ql/authinfo` file, see :ref:`bucket_pw`.
+Unless you have specified the `--plain` option, `mkfs.s3ql` will ask
+you to enter an encryption password. This password will *not* be read
+from an authentication file specified with the :cmdopt:`--authfile`
+option to prevent accidental creation of an encrypted bucket.
+
diff --git a/doc/html/_sources/mount.txt b/doc/html/_sources/mount.txt
index 609c4a4..4f220cd 100644
--- a/doc/html/_sources/mount.txt
+++ b/doc/html/_sources/mount.txt
@@ -22,33 +22,6 @@ This command accepts the following options:
.. pipeinclude:: ../bin/mount.s3ql --help
:start-after: show this help message and exit
-.. _bucket_pw:
-
-Storing Encryption Passwords
-============================
-
-If you are trying to mount an encrypted bucket, `mount.s3ql` will first
-try to read the password from the `.s3ql/authinfo` file (the same file
-that is used to read the backend authentication data) and prompt the
-user to enter the password only if this fails.
-
-The `authinfo` entries to specify bucket passwords are of the form ::
-
- storage-url <storage-url> password <password>
-
-So to always use the password `topsecret` when mounting `s3://joes_bucket`,
-the entry would be ::
-
- storage-url s3://joes_bucket password topsecret
-
-.. NOTE::
-
- If you are using the local backend, the storage url will
- always be converted to an absolute path. So if you are in the
- `/home/john` directory and try to mount `local://bucket`, the matching
- `authinfo` entry has to have a storage url of
- `local:///home/john/bucket`.
-
Compression Algorithms
======================
@@ -83,9 +56,8 @@ Parallel Compression
====================
If you are running S3QL on a system with multiple cores, you might
-want to set ``--compression-threads`` to a value bigger than one. This
-will instruct S3QL to compress and encrypt several blocks at the same
-time.
+want to set the ``--threads`` value larger than one. This will
+instruct S3QL to compress and encrypt several blocks at the same time.
If you want to do this in combination with using the LZMA compression
algorithm, you should keep an eye on memory usage though. Every
diff --git a/doc/html/_sources/special.txt b/doc/html/_sources/special.txt
index c5acade..a217a93 100644
--- a/doc/html/_sources/special.txt
+++ b/doc/html/_sources/special.txt
@@ -11,8 +11,8 @@ Snapshotting and Copy-on-Write
==============================
The command `s3qlcp` can be used to duplicate a directory tree without
-physically copying the file contents. This is possible due to the data
-de-duplication feature of S3QL.
+physically copying the file contents. This is made possible by the
+data de-duplication feature of S3QL.
The syntax of `s3qlcp` is::
diff --git a/doc/html/_sources/tips.txt b/doc/html/_sources/tips.txt
index b857f75..705bc20 100644
--- a/doc/html/_sources/tips.txt
+++ b/doc/html/_sources/tips.txt
@@ -4,8 +4,21 @@
Tips & Tricks
=============
+.. _ssh_tipp:
-.. _copy_performance:
+SSH Backend
+===========
+
+By combining S3QL's local backend with `sshfs
+<http://fuse.sourceforge.net/sshfs.html>`_, it is possible to store an
+S3QL file system on arbitrary SSH servers: first mount the remote
+target directory into the local filesystem, ::
+
+ sshfs user@my.server.com:/mnt/s3ql /mnt/sshfs
+
+and then give the mountpoint to S3QL as a local destination::
+
+ mount.s3ql local:///mnt/sshfs/mybucket /mnt/s3ql
Permanently mounted backup file system
@@ -28,11 +41,16 @@ If you decide to do so, you should make sure to
:cmdopt:`--metadata-upload-interval` option of :program:`mount.s3ql`
to zero).
-
+.. _copy_performance:
Improving copy performance
==========================
+.. NOTE::
+
+ The following applies only when copying data **from** an S3QL file
+ system, **not** when copying data **to** an S3QL file system.
+
If you want to copy a lot of smaller files *from* an S3QL file system
(e.g. for a system restore) you will probably notice that the
performance is rather bad.
@@ -44,6 +62,7 @@ network latency), no matter how big or small the file is. So when you
copy lots of small files, 99% of the time is actually spend waiting
for network data.
+
Theoretically, this problem is easy to solve: you just have to copy
several files at the same time. In practice, however, almost all unix
utilities (``cp``, ``rsync``, ``tar`` and friends) insist on copying
diff --git a/doc/html/about.html b/doc/html/about.html
index f2af2ae..e50b15b 100644
--- a/doc/html/about.html
+++ b/doc/html/about.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>About S3QL &mdash; S3QL 1.0.1 documentation</title>
+ <title>About S3QL &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="#" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Installation" href="installation.html" />
<link rel="prev" title="S3QL User’s Guide" href="index.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="index.html" title="S3QL User’s Guide"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -53,6 +53,7 @@
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -91,12 +92,11 @@
<div class="section" id="about-s3ql">
<h1>About S3QL<a class="headerlink" href="#about-s3ql" title="Permalink to this headline">¶</a></h1>
-<p>S3QL is a file system that stores all its data online. It supports
-<a class="reference external" href="http://aws.amazon.com/s3AmazonS3">Amazon S3</a> as well as arbitrary
-SFTP servers and effectively provides you with a hard disk of dynamic,
-infinite capacity that can be accessed from any computer with internet
-access.</p>
-<p>S3QL is providing a standard, full featured UNIX file system that is
+<p>S3QL is a file system that stores all its data online using storage
+services like <a class="reference external" href="http://code.google.com/apis/storage/">Google Storage</a>, <a class="reference external" href="http://aws.amazon.com/s3AmazonS3">Amazon S3</a> or <a class="reference external" href="http://openstack.org/projects/storage/">OpenStack</a>. S3QL effectively provides
+a hard disk of dynamic, infinite capacity that can be accessed from
+any computer with internet access.</p>
+<p>S3QL is a standard conforming, full featured UNIX file system that is
conceptually indistinguishable from any local file system.
Furthermore, S3QL has additional features like compression,
encryption, data de-duplication, immutable trees and snapshotting
@@ -185,7 +185,7 @@ will, although being inconvenient, not endanger any stored data.</p>
<li class="right" >
<a href="index.html" title="S3QL User’s Guide"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/adm.html b/doc/html/adm.html
index cda4433..64bc811 100644
--- a/doc/html/adm.html
+++ b/doc/html/adm.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Managing Buckets &mdash; S3QL 1.0.1 documentation</title>
+ <title>Managing Buckets &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Mounting" href="mount.html" />
<link rel="prev" title="File System Creation" href="mkfs.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="mkfs.html" title="File System Creation"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="">Managing Buckets</a><ul>
@@ -93,15 +94,15 @@
<div class="section" id="managing-buckets">
<h1>Managing Buckets<a class="headerlink" href="#managing-buckets" title="Permalink to this headline">¶</a></h1>
-<p>The <tt class=" docutils literal"><span class="pre">s3qladm</span></tt> command performs various operations on S3QL buckets.
+<p>The <tt class="docutils literal"><span class="pre">s3qladm</span></tt> command performs various operations on S3QL buckets.
The file system contained in the bucket <em>must not be mounted</em> when
-using <tt class=" docutils literal"><span class="pre">s3qladm</span></tt> or things will go wrong badly.</p>
+using <tt class="docutils literal"><span class="pre">s3qladm</span></tt> or things will go wrong badly.</p>
<p>The syntax is</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3qladm </span><span class="ge">[options]</span><span class="l"> </span><span class="nv">&lt;action&gt;</span><span class="l"> </span><span class="nv">&lt;storage-url&gt;</span><span class="l"></span>
</pre></div>
</div>
<p>where <tt class="var docutils literal"><span class="pre">action</span></tt> may be either of <strong class="program">passphrase</strong>,
-<strong class="program">upgrade</strong>, <strong class="program">delete</strong> or <strong class="program">download-metadata</strong>.</p>
+<strong class="program">upgrade</strong>, <strong class="program">clear</strong> or <strong class="program">download-metadata</strong>.</p>
<p>The <strong class="program">s3qladm</strong> accepts the following general options, no
matter what specific action is being invoked:</p>
<blockquote>
@@ -112,35 +113,38 @@ matter what specific action is being invoked:</p>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to get
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to get
debug messages from all modules. This option can be
specified multiple times.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--quiet</span></kbd></td>
<td>be really quiet</td></tr>
+<tr><td class="option-group">
+<kbd><span class="option">--log <var>&lt;target&gt;</var></span></kbd></td>
+<td>Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify <tt class="docutils literal"><span class="pre">none</span></tt> to disable logging. Default:
+<tt class="docutils literal"><span class="pre">none</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication info.
-(default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default: <tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
<tr><td class="option-group">
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
-<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.</td></tr>
</tbody>
</table>
</div></blockquote>
-<p>Hint: run <tt class=" docutils literal"><span class="pre">s3qladm</span> <span class="pre">&lt;action&gt;</span> <span class="pre">--help</span></tt> to get help on the additional
+<p>Hint: run <tt class="docutils literal"><span class="pre">s3qladm</span> <span class="pre">&lt;action&gt;</span> <span class="pre">--help</span></tt> to get help on the additional
arguments that the different actions take.</p>
<div class="section" id="changing-the-passphrase">
<h2>Changing the Passphrase<a class="headerlink" href="#changing-the-passphrase" title="Permalink to this headline">¶</a></h2>
-<p>To change the passphrase a bucket, use the <tt class=" docutils literal"><span class="pre">s3qladm</span></tt> command:</p>
+<p>To change the passphrase a bucket, use the <tt class="docutils literal"><span class="pre">s3qladm</span></tt> command:</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3qladm passphrase </span><span class="nv">&lt;storage url&gt;</span><span class="l"></span>
</pre></div>
</div>
@@ -164,7 +168,7 @@ execute</p>
<div class="section" id="deleting-a-file-system">
<h2>Deleting a file system<a class="headerlink" href="#deleting-a-file-system" title="Permalink to this headline">¶</a></h2>
<p>A file system can be deleted with:</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3qladm delete </span><span class="nv">&lt;storage url&gt;</span><span class="l"></span>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3qladm clear </span><span class="nv">&lt;storage url&gt;</span><span class="l"></span>
</pre></div>
</div>
<p>This physically deletes all the data and file system structures.</p>
@@ -206,7 +210,7 @@ for help on the mailing list first (see <a class="reference internal" href="reso
<li class="right" >
<a href="mkfs.html" title="File System Creation"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/backends.html b/doc/html/backends.html
index e906681..d810f27 100644
--- a/doc/html/backends.html
+++ b/doc/html/backends.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Storage Backends &mdash; S3QL 1.0.1 documentation</title>
+ <title>Storage Backends &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,9 +26,9 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="File System Creation" href="mkfs.html" />
- <link rel="prev" title="Installation" href="installation.html" />
+ <link rel="prev" title="General Information" href="general.html" />
</head>
<body>
<div class="related">
@@ -38,9 +38,9 @@
<a href="mkfs.html" title="File System Creation"
accesskey="N">next</a></li>
<li class="right" >
- <a href="installation.html" title="Installation"
+ <a href="general.html" title="General Information"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,13 +49,13 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="">Storage Backends</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="#on-backend-reliability">On Backend Reliability</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#the-authinfo-file">The <tt class=" docutils literal"><span class="pre">authinfo</span></tt> file</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#consistency-guarantees">Consistency Guarantees</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#the-amazon-s3-backend">The Amazon S3 Backend</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#the-local-backend">The Local Backend</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#the-sftp-backend">The SFTP Backend</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#google-storage">Google Storage</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#amazon-s3">Amazon S3</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#s3-compatible">S3 compatible</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#local">Local</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#ssh-sftp">SSH/SFTP</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
@@ -94,243 +94,167 @@
<div class="body">
<div class="section" id="storage-backends">
-<h1>Storage Backends<a class="headerlink" href="#storage-backends" title="Permalink to this headline">¶</a></h1>
-<p>S3QL can use different protocols to store the file system data.
-Independent of the backend that you use, the place where your file
-system data is being stored is called a <em>bucket</em>. (This is mostly for
-historical reasons, since initially S3QL supported only the Amazon S3
-backend).</p>
-<div class="section" id="on-backend-reliability">
-<h2>On Backend Reliability<a class="headerlink" href="#on-backend-reliability" title="Permalink to this headline">¶</a></h2>
-<p>S3QL has been designed for use with a storage backend where data loss
-is so infrequent that it can be completely neglected (e.g. the Amazon
-S3 backend). If you decide to use a less reliable backend, you should
-keep the following warning in mind and read this section carefully.</p>
-<div class="admonition warning">
-<p class="first admonition-title">Warning</p>
-<p class="last">S3QL is not able to compensate for any failures of the backend. In
-particular, it is not able reconstruct any data that has been lost
-or corrupted by the backend. The persistence and durability of data
-stored in an S3QL file system is limited and determined by the
-backend alone.</p>
-</div>
-<p>On the plus side, if a backend looses or corrupts some of the stored
-data, S3QL <em>will</em> detect the problem. Missing data will be detected
-when running <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt> or when attempting to access the data in the
-mounted file system. In the later case you will get an IO Error, and
-on unmounting S3QL will warn you that the file system is damaged and
-you need to run <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt>.</p>
-<p><tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt> will report all the affected files and move them into the
-<tt class=" docutils literal"><span class="pre">/lost+found</span></tt> directory of the file system.</p>
-<p>You should be aware that, because of S3QL&#8217;s data de-duplication
-feature, the consequences of a data loss in the backend can be
-significantly more severe than you may expect. More concretely, a data
-loss in the backend at time <em>x</em> may cause data that is written <em>after</em>
-time <em>x</em> to be lost as well. What may happen is this:</p>
-<ol class="arabic simple">
-<li>You store an important file in the S3QL file system.</li>
-<li>The backend looses the data blocks of this file. As long as you
-do not access the file or run <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt>, S3QL
-is not aware that the data has been lost by the backend.</li>
-<li>You save an additional copy of the important file in a different
-location on the same S3QL file system.</li>
-<li>S3QL detects that the contents of the new file are identical to the
-data blocks that have been stored earlier. Since at this point S3QL
-is not aware that these blocks have been lost by the backend, it
-does not save another copy of the file contents in the backend but
-relies on the (presumably) existing blocks instead.</li>
-<li>Therefore, even though you saved another copy, you still do not
-have a backup of the important file (since both copies refer to the
-same data blocks that have been lost by the backend).</li>
-</ol>
-<p>As one can see, this effect becomes the less important the more often
-one runs <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt>, since <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt> will make S3QL aware of any
-blocks that the backend may have lost. Figuratively, this establishes
-a &#8220;checkpoint&#8221;: data loss in the backend that occurred before running
-<tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt> can not affect any file system operations performed after
-running <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt>.</p>
-<p>Nevertheless, (as said at the beginning) the recommended way to use
-S3QL is in combination with a sufficiently reliable storage backend.
-In that case none of the above will ever be a concern.</p>
-</div>
-<div class="section" id="the-authinfo-file">
-<h2>The <tt class=" docutils literal"><span class="pre">authinfo</span></tt> file<a class="headerlink" href="#the-authinfo-file" title="Permalink to this headline">¶</a></h2>
-<p>Most backends first try to read the file <tt class=" docutils literal"><span class="pre">~/.s3ql/authinfo</span></tt> to determine
-the username and password for connecting to the remote host. If this
-fails, both username and password are read from the terminal.</p>
-<p>The <tt class=" docutils literal"><span class="pre">authinfo</span></tt> file has to contain entries of the form</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">backend </span><span class="nv">&lt;backend&gt;</span><span class="l"> machine </span><span class="nv">&lt;host&gt;</span><span class="l"> login </span><span class="nv">&lt;user&gt;</span><span class="l"> password </span><span class="nv">&lt;password&gt;</span><span class="l"></span>
+<span id="id1"></span><h1>Storage Backends<a class="headerlink" href="#storage-backends" title="Permalink to this headline">¶</a></h1>
+<p>The following backends are currently available in S3QL:</p>
+<div class="section" id="google-storage">
+<h2>Google Storage<a class="headerlink" href="#google-storage" title="Permalink to this headline">¶</a></h2>
+<p><a class="reference external" href="http://code.google.com/apis/storage/">Google Storage</a> is an online
+storage service offered by Google. It is the most feature-rich service
+supported by S3QL and S3QL offers the best performance when used with
+the Google Storage backend.</p>
+<p>To use the Google Storage backend, you need to have (or sign up for) a
+Google account, and then <a class="reference external" href="http://code.google.com/apis/storage/docs/signup.html">activate Google Storage</a> for your
+account. The account is free, you will pay only for the amount of
+storage and traffic that you actually use. Once you have created the
+account, make sure to <a class="reference external" href="http://code.google.com/apis/storage/docs/reference/v1/apiversion1.html#enabling">activate legacy access</a>.</p>
+<p>To create a Google Storage bucket, you can use e.g. the <a class="reference external" href="https://sandbox.google.com/storage/">Google
+Storage Manager</a>. The
+storage URL for accessing the bucket in S3QL is then</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">gs://</span><span class="nv">&lt;bucketname&gt;</span><span class="l">/</span><span class="nv">&lt;prefix&gt;</span><span class="l"></span>
</pre></div>
</div>
-<p>So to use the login <tt class=" docutils literal"><span class="pre">joe</span></tt> with password <tt class=" docutils literal"><span class="pre">jibbadup</span></tt> when using the FTP
-backend to connect to the host <tt class=" docutils literal"><span class="pre">backups.joesdomain.com</span></tt>, you would
-specify</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">backend ftp machine backups.joesdomain.com login joe password jibbadup</span>
-</pre></div>
+<p>Here <em>bucketname</em> is the name of the bucket, and <em>prefix</em> can be
+an arbitrary prefix that will be prepended to all object names used by
+S3QL. This allows you to store several S3QL file systems in the same
+Google Storage bucket.</p>
+<p>Note that the backend login and password for accessing your Google
+Storage bucket are not your Google account name and password, but the
+<em>Google Storage developer access key</em> and <em>Google Storage developer
+secret</em> that you can manage with the <a class="reference external" href="https://code.google.com/apis/console/#:storage:legacy">Google Storage key management
+tool</a>.</p>
+<p>If you would like S3QL to connect using HTTPS instead of standard
+HTTP, start the storage url with <tt class="docutils literal"><span class="pre">gss://</span></tt> instead of <tt class="docutils literal"><span class="pre">gs://</span></tt>. Note
+that at this point S3QL does not perform any server certificate
+validation (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=267">issue 267</a>).</p>
</div>
+<div class="section" id="amazon-s3">
+<h2>Amazon S3<a class="headerlink" href="#amazon-s3" title="Permalink to this headline">¶</a></h2>
+<p><a class="reference external" href="http://aws.amazon.com/s3">Amazon S3</a> is the online storage service
+offered by <a class="reference external" href="http://aws.amazon.com/">Amazon Web Services (AWS)</a>. To
+use the S3 backend, you first need to sign up for an AWS account. The
+account is free, you will pay only for the amount of storage and
+traffic that you actually use. After that, you need to create a bucket
+that will hold the S3QL file system, e.g. using the <a class="reference external" href="https://console.aws.amazon.com/s3/home">AWS Management
+Console</a>. For best
+performance, it is recommend to create the bucket in the
+geographically closest storage region, but not the US Standard
+region (see below).</p>
+<p>The storage URL for accessing S3 buckets in S3QL has the form</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3://</span><span class="nv">&lt;bucketname&gt;</span><span class="l">/</span><span class="nv">&lt;prefix&gt;</span><span class="l"></span>
+</pre></div>
</div>
-<div class="section" id="consistency-guarantees">
-<h2>Consistency Guarantees<a class="headerlink" href="#consistency-guarantees" title="Permalink to this headline">¶</a></h2>
-<p>The different backends provide different types of <em>consistency
-guarantees</em>. Informally, a consistency guarantee tells you how fast
-the backend will apply changes to the stored data.</p>
-<p>S3QL defines the following three levels:</p>
-<ul>
-<li><p class="first"><strong>Read-after-Write Consistency.</strong> This is the strongest consistency
-guarantee. If a backend offers read-after-write consistency, it
-guarantees that as soon as you have committed any changes to the
-backend, subsequent requests will take into account these changes.</p>
-</li>
-<li><p class="first"><strong>Read-after-Create Consistency.</strong> If a backend provides only
-read-after-create consistency, only the creation of a new object is
-guaranteed to be taken into account for subsequent requests. This
-means that, for example, if you overwrite data in an existing
-object, subsequent requests may still return the old data for a
-certain period of time.</p>
-</li>
-<li><p class="first"><strong>Eventual consistency.</strong> This is the lowest consistency level.
-Basically, any changes that you make to the backend may not be
-visible for a certain amount of time after the change has been made.
-However, you are guaranteed that no change will be lost. All changes
-will <em>eventually</em> become visible.</p>
-<p>.</p>
-</li>
+<p>Here <em>bucketname</em> is the name of the bucket, and <em>prefix</em> can be
+an arbitrary prefix that will be prepended to all object names used by
+S3QL. This allows you to store several S3QL file systems in the same
+S3 bucket.</p>
+<p>Note that the backend login and password for accessing S3 are not the
+user id and password that you use to log into the Amazon Webpage, but
+the <em>AWS access key id</em> and <em>AWS secret access key</em> shown under <a class="reference external" href="https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&amp;action=access-key">My
+Account/Access Identifiers</a>.</p>
+<p>If you would like S3QL to connect using HTTPS instead of standard
+HTTP, start the storage url with <tt class="docutils literal"><span class="pre">s3s://</span></tt> instead of <tt class="docutils literal"><span class="pre">s3://</span></tt>. Note
+that, as of May 2011, Amazon S3 is faster when accessed using a
+standard HTTP connection, and that S3QL does not perform any server
+certificate validation (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=267">issue 267</a>).</p>
+<div class="section" id="reduced-redundancy-storage-rrs">
+<h3>Reduced Redundancy Storage (RRS)<a class="headerlink" href="#reduced-redundancy-storage-rrs" title="Permalink to this headline">¶</a></h3>
+<p>S3QL does not allow the use of <a class="reference external" href="http://aws.amazon.com/s3/#protecting">reduced redundancy storage</a>. The reason for that is a
+combination of three factors:</p>
+<ul class="simple">
+<li>RRS has a relatively low reliability, on average you loose one
+out of every ten-thousand objects a year. So you can expect to
+occasionally loose some data.</li>
+<li>When <tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt> asks S3 for a list of the stored objects, this list
+includes even those objects that have been lost. Therefore
+<tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt> <em>can not detect lost objects</em> and lost data will only
+become apparent when you try to actually read from a file whose data
+has been lost. This is a (very unfortunate) peculiarity of Amazon
+S3.</li>
+<li>Due to the data de-duplication feature of S3QL, unnoticed lost
+objects may cause subsequent data loss later in time (see
+<a class="reference internal" href="general.html#backend-reliability"><em>On Backend Reliability</em></a> for details).</li>
</ul>
-<p>As long as your backend provides read-after-write or read-after-create
-consistency, you do not have to worry about consistency guarantees at
-all. However, if you plan to use a backend with only eventual
-consistency, you have to be a bit careful in some situations.</p>
-<div class="section" id="dealing-with-eventual-consistency">
-<span id="eventual-consistency"></span><h3>Dealing with Eventual Consistency<a class="headerlink" href="#dealing-with-eventual-consistency" title="Permalink to this headline">¶</a></h3>
-<div class="admonition note">
-<p class="first admonition-title">Note</p>
-<p class="last">The following applies only to storage backends that do not provide
-read-after-create or read-after-write consistency. Currently,
-this is only the Amazon S3 backend <em>if used with the US-Standard
-storage region</em>. If you use a different storage backend, or the S3
-backend with a different storage region, this section does not apply
-to you.</p>
</div>
-<p>While the file system is mounted, S3QL is able to automatically handle
-all issues related to the weak eventual consistency guarantee.
-However, some issues may arise during the mount process and when the
-file system is checked.</p>
+<div class="section" id="potential-issues-when-using-the-us-standard-storage-region">
+<h3>Potential issues when using the US Standard storage region<a class="headerlink" href="#potential-issues-when-using-the-us-standard-storage-region" title="Permalink to this headline">¶</a></h3>
+<p>In the US Standard storage region, Amazon S3 does not guarantee read
+after create consistency. This means that after a new object has been
+stored, requests to read this object may still fail for a little
+while. While the file system is mounted, S3QL is able to automatically
+handle all issues related to this so-called eventual consistency.
+However, problems may arise during the mount process and when the file
+system is checked:</p>
<p>Suppose that you mount the file system, store some new data, delete
-some old data and unmount it again. Now remember that eventual
-consistency means that there is no guarantee that these changes will
-be visible immediately. At least in theory it is therefore possible
-that if you mount the file system again, S3QL does not see any of the
-changes that you have done and presents you an &#8220;old version&#8221; of the
-file system without them. Even worse, if you notice the problem and
-unmount the file system, S3QL will upload the old status (which S3QL
-necessarily has to consider as current) and thereby permanently
-override the newer version (even though this change may not become
-immediately visible either).</p>
-<p>The same problem applies when checking the file system. If the backend
+some old data and unmount it again. Now there is no guarantee that
+these changes will be visible immediately. At least in theory it is
+therefore possible that if you mount the file system again, S3QL
+does not see any of the changes that you have done and presents you
+an &#8220;old version&#8221; of the file system without them. Even worse, if you
+notice the problem and unmount the file system, S3QL will upload the
+old status (which S3QL necessarily has to consider as current) and
+thereby permanently override the newer version (even though this
+change may not become immediately visible either).</p>
+<p>The same problem applies when checking the file system. If S3
provides S3QL with only partially updated data, S3QL has no way to
find out if this a real consistency problem that needs to be fixed or
if it is only a temporary problem that will resolve itself
automatically (because there are still changes that have not become
visible yet).</p>
-<p>While this may seem to be a rather big problem, the likelihood of it
-to occur is rather low. In practice, most storage providers rarely
-need more than a few seconds to apply incoming changes, so to trigger
-this problem one would have to unmount and remount the file system in
-a very short time window. Many people therefore make sure that they
-wait a few minutes between successive mounts (or file system checks)
-and decide that the remaining risk is negligible.</p>
-<p>Nevertheless, the eventual consistency guarantee does not impose an
-upper limit on the time that it may take for change to become visible.
-Therefore there is no &#8220;totally safe&#8221; waiting time that would totally
-eliminate this problem; a theoretical possibility always remains.</p>
+<p>The likelihood of this to happen is rather low. In practice, most
+objects are ready for retrieval just a few seconds after they have
+been stored, so to trigger this problem one would have to unmount and
+remount the file system in a very short time window. However, since S3
+does not place any upper limit on the length of this window, it is
+recommended to not place S3QL buckets in the US Standard storage
+region. As of May 2011, all other storage regions provide stronger
+consistency guarantees that completely eliminate any of the described
+problems.</p>
</div>
</div>
-<div class="section" id="the-amazon-s3-backend">
-<h2>The Amazon S3 Backend<a class="headerlink" href="#the-amazon-s3-backend" title="Permalink to this headline">¶</a></h2>
-<p>To store your file system in an Amazon S3 bucket, use a storage URL of
-the form <tt class=" docutils literal"><span class="pre">s3://&lt;bucketname&gt;</span></tt>. Bucket names must conform to the <a class="reference external" href="http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/BucketRestrictions.html">S3
-Bucket Name Restrictions</a>.</p>
-<p>The S3 backend offers exceptionally strong reliability guarantees. As
-of August 2010, Amazon guarantees a durability of 99.999999999% per
-year. In other words, if you store a thousand million objects then on
-average you would loose less than one object in a hundred years.</p>
-<p>The Amazon S3 backend provides read-after-create consistency for the
-EU, Asia-Pacific and US-West storage regions. <em>For the US-Standard
-storage region, Amazon S3 provides only eventual consistency</em> (please
-refer to <a class="reference internal" href="#eventual-consistency"><em>Dealing with Eventual Consistency</em></a> for information about
-what this entails).</p>
-<p>When connecting to Amazon S3, S3QL uses an unencrypted HTTP
-connection, so if you want your data to stay confidential, you have
-to create the S3QL file system with encryption (this is also the default).</p>
-<p>When reading the authentication information for the S3 backend from
-the <tt class=" docutils literal"><span class="pre">authinfo</span></tt> file, the <tt class=" docutils literal"><span class="pre">host</span></tt> field is ignored, i.e. the first entry
-with <tt class=" docutils literal"><span class="pre">s3</span></tt> as a backend will be used. For example</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">backend s3 machine any login myAWSaccessKeyId password myAwsSecretAccessKey</span>
+<div class="section" id="s3-compatible">
+<h2>S3 compatible<a class="headerlink" href="#s3-compatible" title="Permalink to this headline">¶</a></h2>
+<p>S3QL is also able to access other, S3 compatible storage services for
+which no specific backend exists. Note that when accessing such
+services, only the lowest common denominator of available features can
+be used, so it is generally recommended to use a service specific
+backend instead.</p>
+<p>The storage URL for accessing an arbitrary S3 compatible storage
+service is</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3c://</span><span class="nv">&lt;hostname&gt;</span><span class="l">:</span><span class="nv">&lt;port&gt;</span><span class="l">/</span><span class="nv">&lt;bucketname&gt;</span><span class="l">/</span><span class="nv">&lt;prefix&gt;</span><span class="l"></span>
</pre></div>
</div>
-<p>Note that the bucket names come from a global pool, so chances are
-that your favorite name has already been taken by another S3 user.
-Usually a longer bucket name containing some random numbers, like
-<tt class=" docutils literal"><span class="pre">19283712_yourname_s3ql</span></tt>, will work better.</p>
-<p>If you do not already have one, you need to obtain an Amazon S3
-account from <a class="reference external" href="http://aws.amazon.com/">Amazon AWS</a>. The account is
-free, you will pay only for the amount of storage that you actually
-use.</p>
-<p>Note that the login and password for accessing S3 are not the user id
-and password that you use to log into the Amazon Webpage, but the &#8220;AWS
-access key id&#8221; and &#8220;AWS secret access key&#8221; shown under <a class="reference external" href="https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&amp;action=access-key">My
-Account/Access Identifiers</a>.</p>
-<div class="admonition note">
-<p class="first admonition-title">Note</p>
-<p>S3QL also allows you to use <a class="reference external" href="http://aws.amazon.com/s3/#protecting">reduced redundancy storage</a> by using <tt class="docutils literal"><span class="pre">s3rr://</span></tt>
-instead of <tt class="docutils literal"><span class="pre">s3://</span></tt> in the storage url. However, this not
-recommended. The reason is a combination of three factors:</p>
-<ul class="simple">
-<li>RRS has a relatively low reliability, on average you loose one
-out of every ten-thousand objects a year. So you can expect to
-occasionally loose some data.</li>
-<li>When <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt> asks Amazon S3 for a list of the stored objects,
-this list includes even those objects that have been lost.
-Therefore <tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt> <em>can not detect lost objects</em> and lost data
-will only become apparent when you try to actually read from a
-file whose data has been lost. This is a (very unfortunate)
-peculiarity of Amazon S3.</li>
-<li>Due to the data de-duplication feature of S3QL, unnoticed lost
-objects may cause subsequent data loss later in time (see <a class="reference internal" href="#on-backend-reliability">On
-Backend Reliability</a> for details).</li>
-</ul>
-<p class="last">In other words, you should really only store an S3QL file system
-using RRS if you know exactly what you are getting into.</p>
-</div>
+<p>or</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3cs://</span><span class="nv">&lt;hostname&gt;</span><span class="l">:</span><span class="nv">&lt;port&gt;</span><span class="l">/</span><span class="nv">&lt;bucketname&gt;</span><span class="l">/</span><span class="nv">&lt;prefix&gt;</span><span class="l"></span>
+</pre></div>
</div>
-<div class="section" id="the-local-backend">
-<h2>The Local Backend<a class="headerlink" href="#the-local-backend" title="Permalink to this headline">¶</a></h2>
-<p>The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-<tt class=" docutils literal"><span class="pre">local://&lt;path&gt;</span></tt>. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span class="pre">local:///var/archive</span></tt>.</p>
-<p>The local backend provides read-after-write consistency.</p>
+<p>to use HTTPS connections. Note, however, that at this point S3QL does
+not verify the server certificate (cf. <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=267">issue 267</a>).</p>
</div>
-<div class="section" id="the-sftp-backend">
-<h2>The SFTP Backend<a class="headerlink" href="#the-sftp-backend" title="Permalink to this headline">¶</a></h2>
-<p>The SFTP backend uses the SFTP protocol, which is a file transfer
-protocol similar to ftp, but uses an encrypted SSH connection.
-It provides read-after-write consistency.</p>
-<p>Note that the SFTP backend is rather slow and has not been tested
-as extensively as the S3 and Local backends.</p>
-<p>The storage URL for SFTP connections has the form</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">sftp://</span><span class="nv">&lt;host&gt;</span><span class="ge">[:port]</span><span class="l">/</span><span class="nv">&lt;path&gt;</span><span class="l"></span>
+<div class="section" id="local">
+<h2>Local<a class="headerlink" href="#local" title="Permalink to this headline">¶</a></h2>
+<p>S3QL is also able to store its data on the local file system. This can
+be used to backup data on external media, or to access external
+services that S3QL can not talk to directly (e.g., it is possible to
+store data over SSH by first mounting the remote system using
+<a class="reference external" href="http://fuse.sourceforge.net/sshfs.html">sshfs</a>, then using the local backend to store the data in the sshfs
+mountpoint).</p>
+<p>The storage URL for local storage is</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">local://</span><span class="nv">&lt;path&gt;</span><span class="l"></span>
</pre></div>
</div>
-<p>The SFTP backend will always ask you for a password if you haven&#8217;t
-defined one in <tt class=" docutils literal"><span class="pre">~/.s3ql/authinfo</span></tt>. However, public key authentication
-is tried first and the password will only be used if the public key
-authentication fails.</p>
-<p>The public and private keys will be read from the standard files in
-<tt class=" docutils literal"><span class="pre">~/.ssh/</span></tt>. Note that S3QL will refuse to connect to a computer with
-unknown host key; to add the key to your local keyring you have to
-establish a connection to that computer with the standard SSH command
-line programs first.</p>
+<p>Note that you have to write three consecutive slashes to specify an
+absolute path, e.g. <tt class="docutils literal"><span class="pre">local:///var/archive</span></tt>. Also, relative paths will
+automatically be converted to absolute paths before the authentication
+file is read, i.e. if you are in the <tt class="docutils literal"><span class="pre">/home/john</span></tt> directory and try to
+mount <tt class="docutils literal"><span class="pre">local://bucket</span></tt>, the corresponding section in the
+authentication file must match the storage url
+<tt class="docutils literal"><span class="pre">local:///home/john/bucket</span></tt>.</p>
+</div>
+<div class="section" id="ssh-sftp">
+<h2>SSH/SFTP<a class="headerlink" href="#ssh-sftp" title="Permalink to this headline">¶</a></h2>
+<p>Previous versions of S3QL included an SSH/SFTP backend. With newer
+S3QL versions, it is recommended to instead combine the local backend
+with <a class="reference external" href="http://fuse.sourceforge.net/sshfs.html">sshfs</a> (cf. <a class="reference internal" href="tips.html#ssh-tipp"><em>SSH Backend</em></a>).</p>
</div>
</div>
@@ -347,9 +271,9 @@ line programs first.</p>
<a href="mkfs.html" title="File System Creation"
>next</a></li>
<li class="right" >
- <a href="installation.html" title="Installation"
+ <a href="general.html" title="General Information"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/contrib.html b/doc/html/contrib.html
index cd77857..3238221 100644
--- a/doc/html/contrib.html
+++ b/doc/html/contrib.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Contributed Programs &mdash; S3QL 1.0.1 documentation</title>
+ <title>Contributed Programs &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Tips &amp; Tricks" href="tips.html" />
<link rel="prev" title="Checking for Errors" href="fsck.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="fsck.html" title="Checking for Errors"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -98,12 +99,14 @@
<p>S3QL comes with a few contributed programs that are not part of the
core distribution (and are therefore not installed automatically by
default), but which may nevertheless be useful. These programs are in
-the <tt class=" docutils literal"><span class="pre">contrib</span></tt> directory of the source distribution or in
-<tt class=" docutils literal"><span class="pre">/usr/share/doc/s3ql/contrib</span></tt> if you installed S3QL from a package.</p>
+the <tt class="docutils literal"><span class="pre">contrib</span></tt> directory of the source distribution or in
+<tt class="docutils literal"><span class="pre">/usr/share/doc/s3ql/contrib</span></tt> if you installed S3QL from a package.</p>
<div class="section" id="benchmark-py">
<h2>benchmark.py<a class="headerlink" href="#benchmark-py" title="Permalink to this headline">¶</a></h2>
-<p>This program measures your uplink bandwidth and compression speed and
-recommends a compression algorithm for optimal throughput.</p>
+<p>This program measures S3QL write performance, uplink bandwidth and
+compression speed to determine the limiting factor. It also gives
+recommendation for compression algorithm and number of upload threads
+to achieve maximum performance.</p>
</div>
<div class="section" id="s3-copy-py">
<h2>s3_copy.py<a class="headerlink" href="#s3-copy-py" title="Permalink to this headline">¶</a></h2>
@@ -116,7 +119,7 @@ migrate buckets to a different storage region or storage class
<p><tt class="docutils literal"><span class="pre">pcp.py</span></tt> is a wrapper program that starts several rsync processes to
copy directory trees in parallel. This is important because
transferring files in parallel significantly enhances performance when
-copying data from an S3QL file system (see <a class="reference internal" href="tips.html#copy-performance"><em>Permanently mounted backup file system</em></a> for
+copying data from an S3QL file system (see <a class="reference internal" href="tips.html#copy-performance"><em>Improving copy performance</em></a> for
details).</p>
<p>To recursively copy the directory <tt class="docutils literal"><span class="pre">/mnt/home-backup</span></tt> into
<tt class="docutils literal"><span class="pre">/home/joe</span></tt> using 8 parallel processes and preserving permissions,
@@ -129,7 +132,7 @@ you would execute</p>
<h2>s3_backup.sh<a class="headerlink" href="#s3-backup-sh" title="Permalink to this headline">¶</a></h2>
<p>This is an example script that demonstrates how to set up a simple but
powerful backup solution using S3QL and <a class="reference external" href="http://samba.org/rsync">rsync</a>.</p>
-<p>The <tt class=" docutils literal"><span class="pre">s3_backup.sh</span></tt> script automates the following steps:</p>
+<p>The <tt class="docutils literal"><span class="pre">s3_backup.sh</span></tt> script automates the following steps:</p>
<ol class="arabic simple">
<li>Mount the file system</li>
<li>Replicate the previous backup with <a class="reference internal" href="special.html#s3qlcp"><em>s3qlcp</em></a></li>
@@ -139,7 +142,7 @@ powerful backup solution using S3QL and <a class="reference external" href="http
<li>Unmount the file system</li>
</ol>
<p>The backups are stored in directories of the form
-<tt class=" docutils literal"><span class="pre">YYYY-MM-DD_HH:mm:SS</span></tt> and the <a class="reference internal" href="#expire-backups-py">expire_backups.py</a> command is used to
+<tt class="docutils literal"><span class="pre">YYYY-MM-DD_HH:mm:SS</span></tt> and the <a class="reference internal" href="#expire-backups-py">expire_backups.py</a> command is used to
delete old backups.</p>
</div>
<div class="section" id="expire-backups-py">
@@ -226,7 +229,7 @@ properly unmounts it when the system is shut down.</p>
<li class="right" >
<a href="fsck.html" title="Checking for Errors"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/fsck.html b/doc/html/fsck.html
index df56342..131ecaf 100644
--- a/doc/html/fsck.html
+++ b/doc/html/fsck.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Checking for Errors &mdash; S3QL 1.0.1 documentation</title>
+ <title>Checking for Errors &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Contributed Programs" href="contrib.html" />
<link rel="prev" title="Unmounting" href="umount.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="umount.html" title="Unmounting"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -89,7 +90,7 @@
<h1>Checking for Errors<a class="headerlink" href="#checking-for-errors" title="Permalink to this headline">¶</a></h1>
<p>If, for some reason, the filesystem has not been correctly unmounted,
or if you suspect that there might be errors, you should run the
-<tt class=" docutils literal"><span class="pre">fsck.s3ql</span></tt> utility. It has the following syntax:</p>
+<tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt> utility. It has the following syntax:</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">fsck.s3ql </span><span class="ge">[options]</span><span class="l"> </span><span class="nv">&lt;storage url&gt;</span><span class="l"></span>
</pre></div>
</div>
@@ -99,15 +100,25 @@ or if you suspect that there might be errors, you should run the
<col class="option" />
<col class="description" />
<tbody valign="top">
+<tr><td class="option-group">
+<kbd><span class="option">--log <var>&lt;target&gt;</var></span></kbd></td>
+<td>Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify <tt class="docutils literal"><span class="pre">none</span></tt> to disable logging. Default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/fsck.log</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default: <tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication info.
-(default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to get
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to get
debug messages from all modules. This option can be
specified multiple times.</td></tr>
<tr><td class="option-group">
@@ -117,13 +128,6 @@ specified multiple times.</td></tr>
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext even
-for unencrypted file systems.</td></tr>
-<tr><td class="option-group">
<kbd><span class="option">--batch</span></kbd></td>
<td>If user input is required, exit without prompting.</td></tr>
<tr><td class="option-group">
@@ -149,7 +153,7 @@ for unencrypted file systems.</td></tr>
<li class="right" >
<a href="umount.html" title="Unmounting"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/general.html b/doc/html/general.html
new file mode 100644
index 0000000..a7300d4
--- /dev/null
+++ b/doc/html/general.html
@@ -0,0 +1,251 @@
+
+
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>General Information &mdash; S3QL 1.2 documentation</title>
+
+ <link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '',
+ VERSION: '1.2',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+ <link rel="author" title="About these documents" href="about.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
+ <link rel="next" title="Storage Backends" href="backends.html" />
+ <link rel="prev" title="Installation" href="installation.html" />
+ </head>
+ <body>
+ <div class="related">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="backends.html" title="Storage Backends"
+ accesskey="N">next</a></li>
+ <li class="right" >
+ <a href="installation.html" title="Installation"
+ accesskey="P">previous</a> |</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar">
+ <div class="sphinxsidebarwrapper">
+ <h3><a href="index.html">Table Of Contents</a></h3>
+ <ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
+<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="">General Information</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#terminology">Terminology</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#storing-authentication-information">Storing Authentication Information</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#on-backend-reliability">On Backend Reliability</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
+<li class="toctree-l1"><a class="reference internal" href="mount.html">Mounting</a></li>
+<li class="toctree-l1"><a class="reference internal" href="special.html">Advanced S3QL Features</a></li>
+<li class="toctree-l1"><a class="reference internal" href="umount.html">Unmounting</a></li>
+<li class="toctree-l1"><a class="reference internal" href="fsck.html">Checking for Errors</a></li>
+<li class="toctree-l1"><a class="reference internal" href="contrib.html">Contributed Programs</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tips.html">Tips &amp; Tricks</a></li>
+<li class="toctree-l1"><a class="reference internal" href="issues.html">Known Issues</a></li>
+<li class="toctree-l1"><a class="reference internal" href="man/index.html">Manpages</a></li>
+<li class="toctree-l1"><a class="reference internal" href="resources.html">Further Resources / Getting Help</a></li>
+</ul>
+
+
+ <div id="searchbox" style="display: none">
+ <h3>Quick search</h3>
+ <form class="search" action="search.html" method="get">
+ <input type="text" name="q" size="18" />
+ <input type="submit" value="Go" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+ <p class="searchtip" style="font-size: 90%">
+ Enter search terms.
+ </p>
+ </div>
+ <script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body">
+
+ <div class="section" id="general-information">
+<h1>General Information<a class="headerlink" href="#general-information" title="Permalink to this headline">¶</a></h1>
+<div class="section" id="terminology">
+<h2>Terminology<a class="headerlink" href="#terminology" title="Permalink to this headline">¶</a></h2>
+<p>S3QL can store data at different service providers and using different
+protocols. The term <em>backend</em> refers to both the part of S3QL that
+implements communication with a specific storage service and the
+storage service itself. Most backends can hold more than one S3QL file
+system and thus require some additional information that specifies the
+file system location within the backend. This location is called a
+<em>bucket</em> (for historical reasons).</p>
+<p>Many S3QL commands expect a <em>storage url</em> as a parameter. A storage
+url specifies both the backend and the bucket and thus uniquely
+identifies an S3QL file system. The form of the storage url depends on
+the backend and is described together with the
+<a class="reference internal" href="backends.html#storage-backends"><em>Storage Backends</em></a>.</p>
+</div>
+<div class="section" id="storing-authentication-information">
+<span id="bucket-pw"></span><h2>Storing Authentication Information<a class="headerlink" href="#storing-authentication-information" title="Permalink to this headline">¶</a></h2>
+<p>Normally, S3QL reads username and password for the backend as well as
+an encryption passphrase for the bucket from the terminal. Most
+commands also accept an <tt class="cmdopt docutils literal"><span class="pre">--authfile</span></tt> parameter that can be
+used to read this information from a file instead.</p>
+<p>The authentication file consists of sections, led by a <tt class="docutils literal"><span class="pre">[section]</span></tt>
+header and followed by <tt class="docutils literal"><span class="pre">name:</span> <span class="pre">value</span></tt> entries. The section headers
+themselves are not used by S3QL but have to be unique within the file.</p>
+<p>In each section, the following entries can be defined:</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">storage-url:</th><td class="field-body">Specifies the storage url to which this section applies. If a
+storage url starts with the value of this entry, the section is
+considered applicable.</td>
+</tr>
+<tr class="field-even field"><th class="field-name">backend-login:</th><td class="field-body">Specifies the username to use for authentication with the backend.</td>
+</tr>
+<tr class="field-odd field"><th class="field-name" colspan="2">backend-password:</th></tr>
+<tr class="field-odd field"><td>&nbsp;</td><td class="field-body">Specifies the password to use for authentication with the backend.</td>
+</tr>
+<tr class="field-even field"><th class="field-name" colspan="2">bucket-passphrase:</th></tr>
+<tr class="field-even field"><td>&nbsp;</td><td class="field-body">Specifies the passphrase to use to decrypt the bucket (if it is
+encrypted).</td>
+</tr>
+</tbody>
+</table>
+<p>When reading the authentication file, S3QL considers every applicable
+section in order and uses the last value that it found for each entry.
+For example, consider the following authentication file:</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="ge">[s3]</span><span class="l"></span>
+<span class="l">storage-url: s3://</span>
+<span class="l">backend-login: joe</span>
+<span class="l">backend-password: notquitesecret</span>
+
+<span class="ge">[bucket1]</span><span class="l"></span>
+<span class="l">storage-url: s3://joes-first-bucket</span>
+<span class="l">bucket-passphrase: neitheristhis</span>
+
+<span class="ge">[bucket2]</span><span class="l"></span>
+<span class="l">storage-url: s3://joes-second-bucket</span>
+<span class="l">bucket-passphrase: swordfish</span>
+
+<span class="ge">[bucket3]</span><span class="l"></span>
+<span class="l">storage-url: s3://joes-second-bucket/with-prefix</span>
+<span class="l">backend-login: bill</span>
+<span class="l">backend-password: bi23ll</span>
+<span class="l">bucket-passphrase: ll23bi</span>
+</pre></div>
+</div>
+<p>With this authentication file, S3QL would try to log in as &#8220;joe&#8221;
+whenever the s3 backend is used, except when accessing a storage url
+that begins with &#8220;s3://joes-second-bucket/with-prefix&#8221;. In that case,
+the last section becomes active and S3QL would use the &#8220;bill&#8221;
+credentials. Furthermore, bucket encryption passphrases will be used
+for storage urls that start with &#8220;s3://joes-first-bucket&#8221; or
+&#8220;s3://joes-second-bucket&#8221;.</p>
+<p>The authentication file is parsed by the <a class="reference external" href="http://docs.python.org/library/configparser.html">Python ConfigParser
+module</a>.</p>
+</div>
+<div class="section" id="on-backend-reliability">
+<span id="backend-reliability"></span><h2>On Backend Reliability<a class="headerlink" href="#on-backend-reliability" title="Permalink to this headline">¶</a></h2>
+<p>S3QL has been designed for use with a storage backend where data loss
+is so infrequent that it can be completely neglected (e.g. the Amazon
+S3 backend). If you decide to use a less reliable backend, you should
+keep the following warning in mind and read this section carefully.</p>
+<div class="admonition warning">
+<p class="first admonition-title">Warning</p>
+<p class="last">S3QL is not able to compensate for any failures of the backend. In
+particular, it is not able reconstruct any data that has been lost
+or corrupted by the backend. The persistence and durability of data
+stored in an S3QL file system is limited and determined by the
+backend alone.</p>
+</div>
+<p>On the plus side, if a backend looses or corrupts some of the stored
+data, S3QL <em>will</em> detect the problem. Missing data will be detected
+when running <tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt> or when attempting to access the data in the
+mounted file system. In the later case you will get an IO Error, and
+on unmounting S3QL will warn you that the file system is damaged and
+you need to run <tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt>.</p>
+<p><tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt> will report all the affected files and move them into the
+<tt class="docutils literal"><span class="pre">/lost+found</span></tt> directory of the file system.</p>
+<p>You should be aware that, because of S3QL&#8217;s data de-duplication
+feature, the consequences of a data loss in the backend can be
+significantly more severe than you may expect. More concretely, a data
+loss in the backend at time <em>x</em> may cause data that is written <em>after</em>
+time <em>x</em> to be lost as well. What may happen is this:</p>
+<ol class="arabic simple">
+<li>You store an important file in the S3QL file system.</li>
+<li>The backend looses the data blocks of this file. As long as you
+do not access the file or run <tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt>, S3QL
+is not aware that the data has been lost by the backend.</li>
+<li>You save an additional copy of the important file in a different
+location on the same S3QL file system.</li>
+<li>S3QL detects that the contents of the new file are identical to the
+data blocks that have been stored earlier. Since at this point S3QL
+is not aware that these blocks have been lost by the backend, it
+does not save another copy of the file contents in the backend but
+relies on the (presumably) existing blocks instead.</li>
+<li>Therefore, even though you saved another copy, you still do not
+have a backup of the important file (since both copies refer to the
+same data blocks that have been lost by the backend).</li>
+</ol>
+<p>As one can see, this effect becomes the less important the more often
+one runs <tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt>, since <tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt> will make S3QL aware of any
+blocks that the backend may have lost. Figuratively, this establishes
+a &#8220;checkpoint&#8221;: data loss in the backend that occurred before running
+<tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt> can not affect any file system operations performed after
+running <tt class="docutils literal"><span class="pre">fsck.s3ql</span></tt>.</p>
+<p>Nevertheless, the recommended way to use S3QL is in combination with a
+sufficiently reliable storage backend. In that case none of the above
+will ever be a concern.</p>
+</div>
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="backends.html" title="Storage Backends"
+ >next</a></li>
+ <li class="right" >
+ <a href="installation.html" title="Installation"
+ >previous</a> |</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
+ </ul>
+ </div>
+ <div class="footer">
+ &copy; Copyright 2008-2011, Nikolaus Rath.
+ Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1pre.
+ </div>
+ </body>
+</html> \ No newline at end of file
diff --git a/doc/html/index.html b/doc/html/index.html
index f2e42d1..fa0fc02 100644
--- a/doc/html/index.html
+++ b/doc/html/index.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>S3QL User’s Guide &mdash; S3QL 1.0.1 documentation</title>
+ <title>S3QL User’s Guide &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="#" />
+ <link rel="top" title="S3QL 1.2 documentation" href="#" />
<link rel="next" title="About S3QL" href="about.html" />
</head>
<body>
@@ -36,7 +36,7 @@
<li class="right" style="margin-right: 10px">
<a href="about.html" title="About S3QL"
accesskey="N">next</a></li>
- <li><a href="#">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="#">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -45,6 +45,7 @@
<ul>
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -95,13 +96,18 @@
<li class="toctree-l2"><a class="reference internal" href="installation.html#installing-s3ql">Installing S3QL</a></li>
</ul>
</li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="general.html#terminology">Terminology</a></li>
+<li class="toctree-l2"><a class="reference internal" href="general.html#storing-authentication-information">Storing Authentication Information</a></li>
+<li class="toctree-l2"><a class="reference internal" href="general.html#on-backend-reliability">On Backend Reliability</a></li>
+</ul>
+</li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="backends.html#on-backend-reliability">On Backend Reliability</a></li>
-<li class="toctree-l2"><a class="reference internal" href="backends.html#the-authinfo-file">The <tt class=" docutils literal"><span class="pre">authinfo</span></tt> file</a></li>
-<li class="toctree-l2"><a class="reference internal" href="backends.html#consistency-guarantees">Consistency Guarantees</a></li>
-<li class="toctree-l2"><a class="reference internal" href="backends.html#the-amazon-s3-backend">The Amazon S3 Backend</a></li>
-<li class="toctree-l2"><a class="reference internal" href="backends.html#the-local-backend">The Local Backend</a></li>
-<li class="toctree-l2"><a class="reference internal" href="backends.html#the-sftp-backend">The SFTP Backend</a></li>
+<li class="toctree-l2"><a class="reference internal" href="backends.html#google-storage">Google Storage</a></li>
+<li class="toctree-l2"><a class="reference internal" href="backends.html#amazon-s3">Amazon S3</a></li>
+<li class="toctree-l2"><a class="reference internal" href="backends.html#s3-compatible">S3 compatible</a></li>
+<li class="toctree-l2"><a class="reference internal" href="backends.html#local">Local</a></li>
+<li class="toctree-l2"><a class="reference internal" href="backends.html#ssh-sftp">SSH/SFTP</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
@@ -113,7 +119,6 @@
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="mount.html">Mounting</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="mount.html#storing-encryption-passwords">Storing Encryption Passwords</a></li>
<li class="toctree-l2"><a class="reference internal" href="mount.html#compression-algorithms">Compression Algorithms</a></li>
<li class="toctree-l2"><a class="reference internal" href="mount.html#parallel-compression">Parallel Compression</a></li>
<li class="toctree-l2"><a class="reference internal" href="mount.html#notes-about-caching">Notes about Caching</a></li>
@@ -140,6 +145,7 @@
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="tips.html">Tips &amp; Tricks</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="tips.html#ssh-backend">SSH Backend</a></li>
<li class="toctree-l2"><a class="reference internal" href="tips.html#permanently-mounted-backup-file-system">Permanently mounted backup file system</a></li>
<li class="toctree-l2"><a class="reference internal" href="tips.html#improving-copy-performance">Improving copy performance</a></li>
</ul>
@@ -177,7 +183,7 @@
<li class="right" style="margin-right: 10px">
<a href="about.html" title="About S3QL"
>next</a></li>
- <li><a href="#">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="#">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/installation.html b/doc/html/installation.html
index cb7912b..eaf3225 100644
--- a/doc/html/installation.html
+++ b/doc/html/installation.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Installation &mdash; S3QL 1.0.1 documentation</title>
+ <title>Installation &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,8 +26,8 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
- <link rel="next" title="Storage Backends" href="backends.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
+ <link rel="next" title="General Information" href="general.html" />
<link rel="prev" title="About S3QL" href="about.html" />
</head>
<body>
@@ -35,12 +35,12 @@
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
- <a href="backends.html" title="Storage Backends"
+ <a href="general.html" title="General Information"
accesskey="N">next</a></li>
<li class="right" >
<a href="about.html" title="About S3QL"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -53,6 +53,7 @@
<li class="toctree-l2"><a class="reference internal" href="#installing-s3ql">Installing S3QL</a></li>
</ul>
</li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -108,19 +109,16 @@ running S3QL. Generally, you should first check if your distribution
already provides a suitable packages and only install from source if
that is not the case.</p>
<ul>
-<li><p class="first">Kernel version 2.6.9 or newer. Starting with kernel 2.6.26
-you will get significantly better write performance, so you should
-actually use <em>2.6.26 or newer whenever possible</em>.</p>
-</li>
-<li><p class="first">The <a class="reference external" href="http://fuse.sourceforge.net/">FUSE Library</a> should already be
-installed on your system. However, you have to make sure that you
-have at least version 2.8.0.</p>
+<li><p class="first">Kernel: Linux 2.6.9 or newer or FreeBSD with <a class="reference external" href="http://www.freshports.org/sysutils/fusefs-kmod/">FUSE4BSD</a>. Starting with
+kernel 2.6.26 you will get significantly better write performance,
+so under Linux you should actually use <em>2.6.26 or newer whenever
+possible</em>.</p>
</li>
<li><p class="first">The <a class="reference external" href="http://pypi.python.org/pypi/pycryptopp">PyCrypto++ Python Module</a>. To check if this module
-is installed, try to execute <tt class=" docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">pycryptopp'</span></tt>.</p>
+is installed, try to execute <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">pycryptopp'</span></tt>.</p>
</li>
<li><p class="first">The <a class="reference external" href="http://pypi.python.org/pypi/argparse">argparse Python Module</a>. To check if this module is
-installed, try to execute <tt class=" docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">argparse;</span> <span class="pre">print</span>
+installed, try to execute <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">argparse;</span> <span class="pre">print</span>
<span class="pre">argparse.__version__'</span></tt>. If argparse is installed, this will print
the version number. You need version 1.1 or later.</p>
</li>
@@ -133,23 +131,18 @@ which (if any) version of APWS is installed, run the command</p>
both have to be at least 3.7.0.</p>
</li>
<li><p class="first">The <a class="reference external" href="http://pypi.python.org/pypi/pyliblzma">PyLibLZMA Python module</a>. To check if this module
-is installed, execute <tt class=" docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">lzma;</span> <span class="pre">print</span>
+is installed, execute <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">lzma;</span> <span class="pre">print</span>
<span class="pre">lzma.__version__'</span></tt>. This should print a version number. You need at
least version 0.5.3.</p>
</li>
<li><p class="first">The <a class="reference external" href="http://code.google.com/p/python-llfuse/">Python LLFUSE module</a>. To check if this module
-is installed, execute <tt class=" docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">llfuse;</span> <span class="pre">print</span>
+is installed, execute <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span> <span class="pre">llfuse;</span> <span class="pre">print</span>
<span class="pre">llfuse.__version__'</span></tt>. This should print a version number. You need at
least version 0.29.</p>
<p>Note that earlier S3QL versions shipped with a builtin version of
this module. If you are upgrading from such a version, make sure to
completely remove the old S3QL version first.</p>
</li>
-<li><p class="first">If you want to use the SFTP backend, then you also need the
-<a class="reference external" href="http://www.lag.net/paramiko/">Paramiko Python Module</a>. To check
-if this module is installed, try to execute <tt class=" docutils literal"><span class="pre">python</span> <span class="pre">-c</span> <span class="pre">'import</span>
-<span class="pre">paramiko'</span></tt>.</p>
-</li>
</ul>
</div>
<div class="section" id="installing-s3ql">
@@ -158,18 +151,18 @@ if this module is installed, try to execute <tt class=" docutils literal"><span
<ol class="arabic simple">
<li>Download S3QL from <a class="reference external" href="http://code.google.com/p/s3ql/downloads/list">http://code.google.com/p/s3ql/downloads/list</a></li>
<li>Unpack it into a folder of your choice</li>
-<li>Run <tt class=" docutils literal"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">test</span></tt> to run a self-test. If this fails, ask
+<li>Run <tt class="docutils literal"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">test</span></tt> to run a self-test. If this fails, ask
for help on the <a class="reference external" href="http://groups.google.com/group/s3ql">mailing list</a> or report a bug in the
<a class="reference external" href="http://code.google.com/p/s3ql/issues/list">issue tracker</a>.</li>
</ol>
<p>Now you have three options:</p>
<ul class="simple">
-<li>You can run the S3QL commands from the <tt class=" docutils literal"><span class="pre">bin/</span></tt> directory.</li>
+<li>You can run the S3QL commands from the <tt class="docutils literal"><span class="pre">bin/</span></tt> directory.</li>
<li>You can install S3QL system-wide for all users. To do that, you
-have to run <tt class=" docutils literal"><span class="pre">sudo</span> <span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">install</span></tt>.</li>
-<li>You can install S3QL into <tt class=" docutils literal"><span class="pre">~/.local</span></tt> by executing <tt class=" docutils literal"><span class="pre">python</span>
+have to run <tt class="docutils literal"><span class="pre">sudo</span> <span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">install</span></tt>.</li>
+<li>You can install S3QL into <tt class="docutils literal"><span class="pre">~/.local</span></tt> by executing <tt class="docutils literal"><span class="pre">python</span>
<span class="pre">setup.py</span> <span class="pre">install</span> <span class="pre">--user</span></tt>. In this case you should make sure that
-<tt class=" docutils literal"><span class="pre">~/.local/bin</span></tt> is in your <tt class=" docutils literal"><span class="pre">$PATH</span></tt> environment variable.</li>
+<tt class="docutils literal"><span class="pre">~/.local/bin</span></tt> is in your <tt class="docutils literal"><span class="pre">$PATH</span></tt> environment variable.</li>
</ul>
</div>
</div>
@@ -184,12 +177,12 @@ have to run <tt class=" docutils literal"><span class="pre">sudo</span> <span cl
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
- <a href="backends.html" title="Storage Backends"
+ <a href="general.html" title="General Information"
>next</a></li>
<li class="right" >
<a href="about.html" title="About S3QL"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/issues.html b/doc/html/issues.html
index 3acafe7..0e91509 100644
--- a/doc/html/issues.html
+++ b/doc/html/issues.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Known Issues &mdash; S3QL 1.0.1 documentation</title>
+ <title>Known Issues &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Manpages" href="man/index.html" />
<link rel="prev" title="Tips &amp; Tricks" href="tips.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="tips.html" title="Tips &amp; Tricks"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -88,6 +89,10 @@
<div class="section" id="known-issues">
<h1>Known Issues<a class="headerlink" href="#known-issues" title="Permalink to this headline">¶</a></h1>
<ul>
+<li><p class="first">S3QL does not verify TLS/SSL server certificates, so a
+man-in-the-middle attack is principally possible. See <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=267">issue 267</a> for more
+details.</p>
+</li>
<li><p class="first">S3QL is rather slow when an application tries to write data in
unreasonably small chunks. If a 1 MB file is copied in chunks of 1
KB, this will take more than 10 times as long as when it&#8217;s copied
@@ -95,7 +100,7 @@ with the (recommended) chunk size of 128 KB.</p>
<p>This is a limitation of the FUSE library (which does not yet support
write caching) which will hopefully be addressed in some future FUSE
version.</p>
-<p>Most applications, including e.g. GNU <tt class=" docutils literal"><span class="pre">cp</span></tt> and <tt class=" docutils literal"><span class="pre">rsync</span></tt>, use
+<p>Most applications, including e.g. GNU <tt class="docutils literal"><span class="pre">cp</span></tt> and <tt class="docutils literal"><span class="pre">rsync</span></tt>, use
reasonably large buffers and are therefore not affected by this
problem and perform very efficient on S3QL file systems.</p>
<p>However, if you encounter unexpectedly slow performance with a
@@ -108,15 +113,15 @@ mount option has been specified: the access time (&#8220;atime&#8221;) is only u
if it is currently earlier than either the status change time
(&#8220;ctime&#8221;) or modification time (&#8220;mtime&#8221;).</p>
</li>
-<li><p class="first">S3QL directories always have an <tt class=" docutils literal"><span class="pre">st_nlink</span></tt> value of 1. This may confuse
-programs that rely on directories having <tt class=" docutils literal"><span class="pre">st_nlink</span></tt> values of <em>(2 +
+<li><p class="first">S3QL directories always have an <tt class="docutils literal"><span class="pre">st_nlink</span></tt> value of 1. This may confuse
+programs that rely on directories having <tt class="docutils literal"><span class="pre">st_nlink</span></tt> values of <em>(2 +
number of sub directories)</em>.</p>
<p>Note that this is not a bug in S3QL. Including sub directories in
-the <tt class=" docutils literal"><span class="pre">st_nlink</span></tt> value is a Unix convention, but by no means a
+the <tt class="docutils literal"><span class="pre">st_nlink</span></tt> value is a Unix convention, but by no means a
requirement. If an application blindly relies on this convention
being followed, then this is a bug in the application.</p>
<p>A prominent example are early versions of GNU find, which required
-the <tt class=" docutils literal"><span class="pre">--noleaf</span></tt> option to work correctly on S3QL file systems. This
+the <tt class="docutils literal"><span class="pre">--noleaf</span></tt> option to work correctly on S3QL file systems. This
bug has already been fixed in recent find versions.</p>
</li>
<li><p class="first">In theory, S3QL is not fully compatible with NFS. Since S3QL does
@@ -134,23 +139,23 @@ client 1 knowing about this)</li>
<p>In this situation it is possible that NFS client 1 actually writes
or reads the newly created file B instead. The chances of this are 1
to (2^32 - <em>n</em>) where <em>n</em> is the total number of directory entries
-in the S3QL file system (as displayed by <tt class=" docutils literal"><span class="pre">s3qlstat</span></tt>).</p>
+in the S3QL file system (as displayed by <tt class="docutils literal"><span class="pre">s3qlstat</span></tt>).</p>
<p>Luckily enough, as long as you have less than about 2 thousand
million directory entries (2^31), the chances for this are totally
irrelevant and you don&#8217;t have to worry about it.</p>
</li>
-<li><p class="first">The <tt class=" docutils literal"><span class="pre">umount</span></tt> and <tt class=" docutils literal"><span class="pre">fusermount</span> <span class="pre">-u</span></tt> commands will <em>not</em> block until all
+<li><p class="first">The <tt class="docutils literal"><span class="pre">umount</span></tt> and <tt class="docutils literal"><span class="pre">fusermount</span> <span class="pre">-u</span></tt> commands will <em>not</em> block until all
data has been uploaded to the backend. (this is a FUSE limitation
that will hopefully be removed in the future, see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=159">issue 159</a>). If you use
either command to unmount an S3QL file system, you have to take care
-to explicitly wait for the <tt class=" docutils literal"><span class="pre">mount.s3ql</span></tt> process to terminate before
+to explicitly wait for the <tt class="docutils literal"><span class="pre">mount.s3ql</span></tt> process to terminate before
you shut down or restart the system. Therefore it is generally not a
-good idea to mount an S3QL file system in <tt class=" docutils literal"><span class="pre">/etc/fstab</span></tt> (you should
+good idea to mount an S3QL file system in <tt class="docutils literal"><span class="pre">/etc/fstab</span></tt> (you should
use a dedicated init script instead).</p>
</li>
<li><p class="first">S3QL relies on the backends not to run out of space. This is a given
for big storage providers like Amazon S3, but you may stumble upon
-this if you store buckets e.g. on a small sftp server.</p>
+this if you store buckets e.g. on smaller servers or servies.</p>
<p>If there is no space left in the backend, attempts to write more
data into the S3QL file system will fail and the file system will be
in an inconsistent state and require a file system check (and you
@@ -180,7 +185,7 @@ the backend.</p>
<li class="right" >
<a href="tips.html" title="Tips &amp; Tricks"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/man/adm.html b/doc/html/man/adm.html
index 7345230..a28cfbc 100644
--- a/doc/html/man/adm.html
+++ b/doc/html/man/adm.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The s3qladm command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The s3qladm command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The mount.s3ql command" href="mount.html" />
<link rel="prev" title="The mkfs.s3ql command" href="mkfs.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="mkfs.html" title="The mkfs.s3ql command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -116,28 +117,8 @@
<p>The <strong class="program">s3qladm</strong> command performs various operations on S3QL buckets.
The file system contained in the bucket <em>must not be mounted</em> when
using <strong class="program">s3qladm</strong> or things will go wrong badly.</p>
-<p>The form of the storage url depends on the backend that is used. The
-following backends are supported:</p>
-<div class="section" id="amazon-s3">
-<h3>Amazon S3<a class="headerlink" href="#amazon-s3" title="Permalink to this headline">¶</a></h3>
-<p>To store your file system in an Amazon S3 bucket, use a storage URL of
-the form <tt class=" docutils literal"><span class="pre">s3://&lt;bucketname&gt;</span></tt>. Bucket names must conform to the S3 Bucket
-Name Restrictions.</p>
-</div>
-<div class="section" id="local">
-<h3>Local<a class="headerlink" href="#local" title="Permalink to this headline">¶</a></h3>
-<p>The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-<tt class=" docutils literal"><span class="pre">local://&lt;path&gt;</span></tt>. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span class="pre">local:///var/archive</span></tt>.</p>
-</div>
-<div class="section" id="sftp">
-<h3>SFTP<a class="headerlink" href="#sftp" title="Permalink to this headline">¶</a></h3>
-<p>The storage URL for SFTP connections has the form</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">sftp://</span><span class="nv">&lt;host&gt;</span><span class="ge">[:port]</span><span class="l">/</span><span class="nv">&lt;path&gt;</span><span class="l"></span>
-</pre></div>
-</div>
-</div>
+<p>The storage url depends on the backend that is used. The S3QL User&#8217;s
+Guide should be consulted for a description of the available backends.</p>
</div>
<div class="section" id="options">
<h2>Options<a class="headerlink" href="#options" title="Permalink to this headline">¶</a></h2>
@@ -150,31 +131,34 @@ slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span cl
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to get
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to get
debug messages from all modules. This option can be
specified multiple times.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--quiet</span></kbd></td>
<td>be really quiet</td></tr>
+<tr><td class="option-group">
+<kbd><span class="option">--log <var>&lt;target&gt;</var></span></kbd></td>
+<td>Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify <tt class="docutils literal"><span class="pre">none</span></tt> to disable logging. Default:
+<tt class="docutils literal"><span class="pre">none</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication info.
-(default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default: <tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
<tr><td class="option-group">
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
-<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.</td></tr>
</tbody>
</table>
</div></blockquote>
-<p>Hint: run <tt class=" docutils literal"><span class="pre">s3qladm</span> <span class="pre">&lt;action&gt;</span> <span class="pre">--help</span></tt> to get help on the additional
+<p>Hint: run <tt class="docutils literal"><span class="pre">s3qladm</span> <span class="pre">&lt;action&gt;</span> <span class="pre">--help</span></tt> to get help on the additional
arguments that the different actions take.</p>
</div>
<div class="section" id="actions">
@@ -191,13 +175,6 @@ arguments that the different actions take.</p>
<dd>Interactively download backups of the file system metadata.</dd>
</dl>
</div>
-<div class="section" id="files">
-<h2>Files<a class="headerlink" href="#files" title="Permalink to this headline">¶</a></h2>
-<p>Authentication data for backends and bucket encryption passphrases are
-read from <tt class="file docutils literal"><span class="pre">authinfo</span></tt> in <tt class="file docutils literal"><span class="pre">~/.s3ql</span></tt> or the directory
-specified with <tt class="cmdopt docutils literal"><span class="pre">--homedir</span></tt>. Log files are placed in the same
-directory.</p>
-</div>
<div class="section" id="exit-status">
<h2>Exit Status<a class="headerlink" href="#exit-status" title="Permalink to this headline">¶</a></h2>
<p><strong class="program">s3qladm</strong> returns exit code 0 if the operation succeeded and 1 if some
@@ -227,7 +204,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="mkfs.html" title="The mkfs.s3ql command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/cp.html b/doc/html/man/cp.html
index bfc9248..c2e8e0c 100644
--- a/doc/html/man/cp.html
+++ b/doc/html/man/cp.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The s3qlcp command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The s3qlcp command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The s3qlrm command" href="rm.html" />
<link rel="prev" title="The s3qlctrl command" href="ctrl.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="ctrl.html" title="The s3qlctrl command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -117,30 +118,26 @@ Both source and destination must lie inside the same S3QL file system.</p>
<p>The replication will not take any additional space. Only if one of
directories is modified later on, the modified data will take
additional storage space.</p>
-<p><tt class=" docutils literal"><span class="pre">s3qlcp</span></tt> can only be called by the user that mounted the file system
-and (if the file system was mounted with <tt class=" docutils literal"><span class="pre">--allow-other</span></tt> or <tt class=" docutils literal"><span class="pre">--allow-root</span></tt>)
+<p><tt class="docutils literal"><span class="pre">s3qlcp</span></tt> can only be called by the user that mounted the file system
+and (if the file system was mounted with <tt class="docutils literal"><span class="pre">--allow-other</span></tt> or <tt class="docutils literal"><span class="pre">--allow-root</span></tt>)
the root user. This limitation might be removed in the future (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=155">issue 155</a>).</p>
<p>Note that:</p>
<ul class="simple">
<li>After the replication, both source and target directory will still
-be completely ordinary directories. You can regard <tt class=" docutils literal"><span class="pre">&lt;src&gt;</span></tt> as a
-snapshot of <tt class=" docutils literal"><span class="pre">&lt;target&gt;</span></tt> or vice versa. However, the most common
-usage of <tt class=" docutils literal"><span class="pre">s3qlcp</span></tt> is to regularly duplicate the same source
-directory, say <tt class=" docutils literal"><span class="pre">documents</span></tt>, to different target directories. For a
+be completely ordinary directories. You can regard <tt class="docutils literal"><span class="pre">&lt;src&gt;</span></tt> as a
+snapshot of <tt class="docutils literal"><span class="pre">&lt;target&gt;</span></tt> or vice versa. However, the most common
+usage of <tt class="docutils literal"><span class="pre">s3qlcp</span></tt> is to regularly duplicate the same source
+directory, say <tt class="docutils literal"><span class="pre">documents</span></tt>, to different target directories. For a
e.g. monthly replication, the target directories would typically be
-named something like <tt class=" docutils literal"><span class="pre">documents_Januray</span></tt> for the replication in
-January, <tt class=" docutils literal"><span class="pre">documents_February</span></tt> for the replication in February etc.
+named something like <tt class="docutils literal"><span class="pre">documents_January</span></tt> for the replication in
+January, <tt class="docutils literal"><span class="pre">documents_February</span></tt> for the replication in February etc.
In this case it is clear that the target directories should be
regarded as snapshots of the source directory.</li>
<li>Exactly the same effect could be achieved by an ordinary copy
-program like <tt class=" docutils literal"><span class="pre">cp</span> <span class="pre">-a</span></tt>. However, this procedure would be orders of
-magnitude slower, because <tt class=" docutils literal"><span class="pre">cp</span></tt> would have to read every file
+program like <tt class="docutils literal"><span class="pre">cp</span> <span class="pre">-a</span></tt>. However, this procedure would be orders of
+magnitude slower, because <tt class="docutils literal"><span class="pre">cp</span></tt> would have to read every file
completely (so that S3QL had to fetch all the data over the network
from the backend) before writing them into the destination folder.</li>
-<li>Before starting with the replication, S3QL has to flush the local
-cache. So if you just copied lots of new data into the file system
-that has not yet been uploaded, replication will take longer than
-usual.</li>
</ul>
<div class="section" id="snapshotting-vs-hardlinking">
<h3>Snapshotting vs Hardlinking<a class="headerlink" href="#snapshotting-vs-hardlinking" title="Permalink to this headline">¶</a></h3>
@@ -213,7 +210,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="ctrl.html" title="The s3qlctrl command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/ctrl.html b/doc/html/man/ctrl.html
index b6acd2e..adbcb74 100644
--- a/doc/html/man/ctrl.html
+++ b/doc/html/man/ctrl.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The s3qlctrl command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The s3qlctrl command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The s3qlcp command" href="cp.html" />
<link rel="prev" title="The s3qlstat command" href="stat.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="stat.html" title="The s3qlstat command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -116,6 +117,10 @@
<h2>Description<a class="headerlink" href="#description" title="Permalink to this headline">¶</a></h2>
<p>The <strong class="command">s3qlctrl</strong> command performs various actions on the S3QL file system mounted
in <tt class="var docutils literal"><span class="pre">mountpoint</span></tt>.</p>
+<p><strong class="command">s3qlctrl</strong> can only be called by the user that mounted the file system
+and (if the file system was mounted with <tt class="cmdopt docutils literal"><span class="pre">--allow-other</span></tt> or
+<tt class="cmdopt docutils literal"><span class="pre">--allow-root</span></tt>) the root user. This limitation might be
+removed in the future (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=155">issue 155</a>).</p>
<p>The following actions may be specified:</p>
<dl class="docutils">
<dt>flushcache</dt>
@@ -164,7 +169,7 @@ what specific action is being invoked:</p>
</tbody>
</table>
</div></blockquote>
-<p>Hint: run <tt class=" docutils literal"><span class="pre">s3qlctrl</span> <span class="pre">&lt;action&gt;</span> <span class="pre">--help</span></tt> to get help on the additional
+<p>Hint: run <tt class="docutils literal"><span class="pre">s3qlctrl</span> <span class="pre">&lt;action&gt;</span> <span class="pre">--help</span></tt> to get help on the additional
arguments that the different actions take.</p>
</div>
<div class="section" id="exit-status">
@@ -196,7 +201,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="stat.html" title="The s3qlstat command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/expire_backups.html b/doc/html/man/expire_backups.html
index e50daa6..4ebdbc8 100644
--- a/doc/html/man/expire_backups.html
+++ b/doc/html/man/expire_backups.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The expire_backups command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The expire_backups command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="Further Resources / Getting Help" href="../resources.html" />
<link rel="prev" title="The pcp command" href="pcp.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="pcp.html" title="The pcp command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -199,7 +200,7 @@ tamper with the state file.</p>
dates.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--use-s3qlrm</span></kbd></td>
-<td>Use <tt class=" docutils literal"><span class="pre">s3qlrm</span></tt> command to delete backups.</td></tr>
+<td>Use <tt class="docutils literal"><span class="pre">s3qlrm</span></tt> command to delete backups.</td></tr>
</tbody>
</table>
</div></blockquote>
@@ -230,7 +231,7 @@ error occured.</p>
<li class="right" >
<a href="pcp.html" title="The pcp command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/fsck.html b/doc/html/man/fsck.html
index 363ea51..875d0a5 100644
--- a/doc/html/man/fsck.html
+++ b/doc/html/man/fsck.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The fsck.s3ql command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The fsck.s3ql command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The pcp command" href="pcp.html" />
<link rel="prev" title="The umount.s3ql command" href="umount.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="umount.html" title="The umount.s3ql command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -113,29 +114,9 @@
<h2>Description<a class="headerlink" href="#description" title="Permalink to this headline">¶</a></h2>
<p>The <strong class="command">mkfs.s3ql</strong> command checks the new file system in the location
specified by <em>storage url</em> for errors and attempts to repair any
-problems.</p>
-<p>The form of the storage url depends on the backend that is used. The
-following backends are supported:</p>
-<div class="section" id="amazon-s3">
-<h3>Amazon S3<a class="headerlink" href="#amazon-s3" title="Permalink to this headline">¶</a></h3>
-<p>To store your file system in an Amazon S3 bucket, use a storage URL of
-the form <tt class=" docutils literal"><span class="pre">s3://&lt;bucketname&gt;</span></tt>. Bucket names must conform to the S3 Bucket
-Name Restrictions.</p>
-</div>
-<div class="section" id="local">
-<h3>Local<a class="headerlink" href="#local" title="Permalink to this headline">¶</a></h3>
-<p>The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-<tt class=" docutils literal"><span class="pre">local://&lt;path&gt;</span></tt>. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span class="pre">local:///var/archive</span></tt>.</p>
-</div>
-<div class="section" id="sftp">
-<h3>SFTP<a class="headerlink" href="#sftp" title="Permalink to this headline">¶</a></h3>
-<p>The storage URL for SFTP connections has the form</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">sftp://</span><span class="nv">&lt;host&gt;</span><span class="ge">[:port]</span><span class="l">/</span><span class="nv">&lt;path&gt;</span><span class="l"></span>
-</pre></div>
-</div>
-</div>
+problems. The storage url depends on the backend that is used. The
+S3QL User&#8217;s Guide should be consulted for a description of the
+available backends.</p>
</div>
<div class="section" id="options">
<h2>Options<a class="headerlink" href="#options" title="Permalink to this headline">¶</a></h2>
@@ -145,15 +126,25 @@ slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span cl
<col class="option" />
<col class="description" />
<tbody valign="top">
+<tr><td class="option-group">
+<kbd><span class="option">--log <var>&lt;target&gt;</var></span></kbd></td>
+<td>Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify <tt class="docutils literal"><span class="pre">none</span></tt> to disable logging. Default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/fsck.log</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default: <tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication info.
-(default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to get
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to get
debug messages from all modules. This option can be
specified multiple times.</td></tr>
<tr><td class="option-group">
@@ -163,13 +154,6 @@ specified multiple times.</td></tr>
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext even
-for unencrypted file systems.</td></tr>
-<tr><td class="option-group">
<kbd><span class="option">--batch</span></kbd></td>
<td>If user input is required, exit without prompting.</td></tr>
<tr><td class="option-group">
@@ -179,13 +163,6 @@ for unencrypted file systems.</td></tr>
</table>
</div></blockquote>
</div>
-<div class="section" id="files">
-<h2>Files<a class="headerlink" href="#files" title="Permalink to this headline">¶</a></h2>
-<p>Authentication data for backends and bucket encryption passphrases are
-read from <tt class="file docutils literal"><span class="pre">authinfo</span></tt> in <tt class="file docutils literal"><span class="pre">~/.s3ql</span></tt> or the directory
-specified with <tt class="cmdopt docutils literal"><span class="pre">--homedir</span></tt>. Log files are placed in the same
-directory.</p>
-</div>
<div class="section" id="exit-status">
<h2>Exit Status<a class="headerlink" href="#exit-status" title="Permalink to this headline">¶</a></h2>
<p><strong class="command">mkfs.s3ql</strong> returns exit code 0 if the operation succeeded and 1 if some
@@ -215,7 +192,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="umount.html" title="The umount.s3ql command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/index.html b/doc/html/man/index.html
index a3141b9..8b5c000 100644
--- a/doc/html/man/index.html
+++ b/doc/html/man/index.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Manpages &mdash; S3QL 1.0.1 documentation</title>
+ <title>Manpages &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="next" title="The mkfs.s3ql command" href="mkfs.html" />
<link rel="prev" title="Known Issues" href="../issues.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="../issues.html" title="Known Issues"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -137,7 +138,7 @@ here in the User&#8217;s Guide.</p>
<li class="right" >
<a href="../issues.html" title="Known Issues"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/man/lock.html b/doc/html/man/lock.html
index acb65da..36a08d0 100644
--- a/doc/html/man/lock.html
+++ b/doc/html/man/lock.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The s3qllock command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The s3qllock command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The umount.s3ql command" href="umount.html" />
<link rel="prev" title="The s3qlrm command" href="rm.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="rm.html" title="The s3qlrm command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -116,6 +117,10 @@ system immutable. Immutable trees can no longer be changed in any way
whatsoever. You can not add new files or directories and you can not
change or delete existing files and directories. The only way to get
rid of an immutable tree is to use the <strong class="program">s3qlrm</strong> command.</p>
+<p><strong class="command">s3qllock</strong> can only be called by the user that mounted the file system
+and (if the file system was mounted with <tt class="cmdopt docutils literal"><span class="pre">--allow-other</span></tt> or
+<tt class="cmdopt docutils literal"><span class="pre">--allow-root</span></tt>) the root user. This limitation might be
+removed in the future (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=155">issue 155</a>).</p>
</div>
<div class="section" id="rationale">
<h2>Rationale<a class="headerlink" href="#rationale" title="Permalink to this headline">¶</a></h2>
@@ -196,7 +201,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="rm.html" title="The s3qlrm command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/mkfs.html b/doc/html/man/mkfs.html
index 8117601..3047249 100644
--- a/doc/html/man/mkfs.html
+++ b/doc/html/man/mkfs.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The mkfs.s3ql command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The mkfs.s3ql command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The s3qladm command" href="adm.html" />
<link rel="prev" title="Manpages" href="index.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="index.html" title="Manpages"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -112,29 +113,13 @@
<div class="section" id="description">
<h2>Description<a class="headerlink" href="#description" title="Permalink to this headline">¶</a></h2>
<p>The <strong class="command">mkfs.s3ql</strong> command creates a new file system in the location
-specified by <em>storage url</em>.</p>
-<p>The form of the storage url depends on the backend that is used. The
-following backends are supported:</p>
-<div class="section" id="amazon-s3">
-<h3>Amazon S3<a class="headerlink" href="#amazon-s3" title="Permalink to this headline">¶</a></h3>
-<p>To store your file system in an Amazon S3 bucket, use a storage URL of
-the form <tt class=" docutils literal"><span class="pre">s3://&lt;bucketname&gt;</span></tt>. Bucket names must conform to the S3 Bucket
-Name Restrictions.</p>
-</div>
-<div class="section" id="local">
-<h3>Local<a class="headerlink" href="#local" title="Permalink to this headline">¶</a></h3>
-<p>The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-<tt class=" docutils literal"><span class="pre">local://&lt;path&gt;</span></tt>. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span class="pre">local:///var/archive</span></tt>.</p>
-</div>
-<div class="section" id="sftp">
-<h3>SFTP<a class="headerlink" href="#sftp" title="Permalink to this headline">¶</a></h3>
-<p>The storage URL for SFTP connections has the form</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">sftp://</span><span class="nv">&lt;host&gt;</span><span class="ge">[:port]</span><span class="l">/</span><span class="nv">&lt;path&gt;</span><span class="l"></span>
-</pre></div>
-</div>
-</div>
+specified by <em>storage url</em>. The storage url depends on the backend
+that is used. The S3QL User&#8217;s Guide should be consulted for a
+description of the available backends.</p>
+<p>Unless you have specified the <tt class="docutils literal"><span class="pre">--plain</span></tt> option, <tt class="docutils literal"><span class="pre">mkfs.s3ql</span></tt> will ask
+you to enter an encryption password. This password will <em>not</em> be read
+from an authentication file specified with the <tt class="cmdopt docutils literal"><span class="pre">--authfile</span></tt>
+option to prevent accidental creation of an encrypted bucket.</p>
</div>
<div class="section" id="options">
<h2>Options<a class="headerlink" href="#options" title="Permalink to this headline">¶</a></h2>
@@ -145,16 +130,20 @@ slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span cl
<col class="description" />
<tbody valign="top">
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication
-info. (default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default: <tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to
-get debug messages from all modules. This option can
-be specified multiple times.</td></tr>
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to
+get debug messages from all modules. This option can be
+specified multiple times.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--quiet</span></kbd></td>
<td>be really quiet</td></tr>
@@ -162,19 +151,6 @@ be specified multiple times.</td></tr>
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.</td></tr>
-<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--s3-location <var>&lt;name&gt;</var></span></kbd></td>
-</tr>
-<tr><td>&nbsp;</td><td>Storage location for new S3 buckets. Allowed values:
-<tt class=" docutils literal"><span class="pre">EU</span></tt>, <tt class=" docutils literal"><span class="pre">us-west-1</span></tt>, <tt class=" docutils literal"><span class="pre">ap-southeast-1</span></tt>, or <tt class=" docutils literal"><span class="pre">us-standard</span></tt>.
-(default: EU)</td></tr>
-<tr><td class="option-group">
<kbd><span class="option">-L <var>&lt;name&gt;</var></span></kbd></td>
<td>Filesystem label</td></tr>
<tr><td class="option-group" colspan="2">
@@ -191,13 +167,6 @@ even for unencrypted file systems.</td></tr>
</table>
</div></blockquote>
</div>
-<div class="section" id="files">
-<h2>Files<a class="headerlink" href="#files" title="Permalink to this headline">¶</a></h2>
-<p>Authentication data for backends and bucket encryption passphrases are
-read from <tt class="file docutils literal"><span class="pre">authinfo</span></tt> in <tt class="file docutils literal"><span class="pre">~/.s3ql</span></tt> or the directory
-specified with <tt class="cmdopt docutils literal"><span class="pre">--homedir</span></tt>. Log files are placed in the same
-directory.</p>
-</div>
<div class="section" id="exit-status">
<h2>Exit Status<a class="headerlink" href="#exit-status" title="Permalink to this headline">¶</a></h2>
<p><strong class="command">mkfs.s3ql</strong> returns exit code 0 if the operation succeeded and 1 if some
@@ -227,7 +196,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="index.html" title="Manpages"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/mount.html b/doc/html/man/mount.html
index 5f91bc6..2873b2d 100644
--- a/doc/html/man/mount.html
+++ b/doc/html/man/mount.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The mount.s3ql command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The mount.s3ql command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The s3qlstat command" href="stat.html" />
<link rel="prev" title="The s3qladm command" href="adm.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="adm.html" title="The s3qladm command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -112,29 +113,9 @@
<div class="section" id="description">
<h2>Description<a class="headerlink" href="#description" title="Permalink to this headline">¶</a></h2>
<p>The <strong class="command">mount.s3ql</strong> command mounts the S3QL file system stored in <em>storage
-url</em> in the directory <em>mount point</em>.</p>
-<p>The form of the storage url depends on the backend that is used. The
-following backends are supported:</p>
-<div class="section" id="amazon-s3">
-<h3>Amazon S3<a class="headerlink" href="#amazon-s3" title="Permalink to this headline">¶</a></h3>
-<p>To store your file system in an Amazon S3 bucket, use a storage URL of
-the form <tt class=" docutils literal"><span class="pre">s3://&lt;bucketname&gt;</span></tt>. Bucket names must conform to the S3 Bucket
-Name Restrictions.</p>
-</div>
-<div class="section" id="local">
-<h3>Local<a class="headerlink" href="#local" title="Permalink to this headline">¶</a></h3>
-<p>The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-<tt class=" docutils literal"><span class="pre">local://&lt;path&gt;</span></tt>. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span class="pre">local:///var/archive</span></tt>.</p>
-</div>
-<div class="section" id="sftp">
-<h3>SFTP<a class="headerlink" href="#sftp" title="Permalink to this headline">¶</a></h3>
-<p>The storage URL for SFTP connections has the form</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">sftp://</span><span class="nv">&lt;host&gt;</span><span class="ge">[:port]</span><span class="l">/</span><span class="nv">&lt;path&gt;</span><span class="l"></span>
-</pre></div>
-</div>
-</div>
+url</em> in the directory <em>mount point</em>. The storage url depends on the
+backend that is used. The S3QL User&#8217;s Guide should be consulted for a
+description of the available backends.</p>
</div>
<div class="section" id="options">
<h2>Options<a class="headerlink" href="#options" title="Permalink to this headline">¶</a></h2>
@@ -144,15 +125,26 @@ slashes to specify an absolute path, e.g. <tt class=" docutils literal"><span cl
<col class="option" />
<col class="description" />
<tbody valign="top">
+<tr><td class="option-group">
+<kbd><span class="option">--log <var>&lt;target&gt;</var></span></kbd></td>
+<td>Write logging info into this file. File will be
+rotated when it reaches 1 MB, and at most 5 old log
+files will be kept. Specify <tt class="docutils literal"><span class="pre">none</span></tt> to disable
+logging. Default: <tt class="docutils literal"><span class="pre">~/.s3ql/mount.log</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication
-info. (default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file
+(default: <tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to
get debug messages from all modules. This option can
be specified multiple times.</td></tr>
<tr><td class="option-group">
@@ -161,13 +153,6 @@ be specified multiple times.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
-<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--cachesize <var>&lt;size&gt;</var></span></kbd></td>
</tr>
@@ -183,20 +168,20 @@ operation.</td></tr>
Each cache entry requires one file descriptor, so if
you increase this number you have to make sure that
your process file descriptor limit (as set with
-<tt class=" docutils literal"><span class="pre">ulimit</span> <span class="pre">-n</span></tt>) is high enough (at least the number of
+<tt class="docutils literal"><span class="pre">ulimit</span> <span class="pre">-n</span></tt>) is high enough (at least the number of
cache entries + 100).</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--allow-other</span></kbd></td>
-<td>Normally, only the user who called <tt class=" docutils literal"><span class="pre">mount.s3ql</span></tt> can
+<td>Normally, only the user who called <tt class="docutils literal"><span class="pre">mount.s3ql</span></tt> can
access the mount point. This user then also has full
access to it, independent of individual file
-permissions. If the <tt class=" docutils literal"><span class="pre">--allow-other</span></tt> option is
+permissions. If the <tt class="docutils literal"><span class="pre">--allow-other</span></tt> option is
specified, other users can access the mount point as
well and individual file permissions are taken into
account for all users.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--allow-root</span></kbd></td>
-<td>Like <tt class=" docutils literal"><span class="pre">--allow-other</span></tt>, but restrict access to the
+<td>Like <tt class="docutils literal"><span class="pre">--allow-other</span></tt>, but restrict access to the
mounting user and the root user.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--fg</span></kbd></td>
@@ -217,29 +202,25 @@ this, then you don&#8217;t need it.</td></tr>
<kbd><span class="option">--compress <var>&lt;name&gt;</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Compression algorithm to use when storing new data.
-Allowed values: <tt class=" docutils literal"><span class="pre">lzma</span></tt>, <tt class=" docutils literal"><span class="pre">bzip2</span></tt>, <tt class=" docutils literal"><span class="pre">zlib</span></tt>, none.
-(default: <tt class=" docutils literal"><span class="pre">lzma</span></tt>)</td></tr>
+Allowed values: <tt class="docutils literal"><span class="pre">lzma</span></tt>, <tt class="docutils literal"><span class="pre">bzip2</span></tt>, <tt class="docutils literal"><span class="pre">zlib</span></tt>, none.
+(default: <tt class="docutils literal"><span class="pre">lzma</span></tt>)</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--metadata-upload-interval <var>&lt;seconds&gt;</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Interval in seconds between complete metadata uploads.
Set to 0 to disable. Default: 24h.</td></tr>
-<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--compression-threads <var>&lt;no&gt;</var></span></kbd></td>
-</tr>
-<tr><td>&nbsp;</td><td>Number of parallel compression and encryption threads
-to use (default: 1).</td></tr>
+<tr><td class="option-group">
+<kbd><span class="option">--threads <var>&lt;no&gt;</var></span></kbd></td>
+<td>Number of parallel upload threads to use (default:
+auto).</td></tr>
+<tr><td class="option-group">
+<kbd><span class="option">--nfs</span></kbd></td>
+<td>Support export of S3QL file systems over NFS (default:
+False)</td></tr>
</tbody>
</table>
</div></blockquote>
</div>
-<div class="section" id="files">
-<h2>Files<a class="headerlink" href="#files" title="Permalink to this headline">¶</a></h2>
-<p>Authentication data for backends and bucket encryption passphrases are
-read from <tt class="file docutils literal"><span class="pre">authinfo</span></tt> in <tt class="file docutils literal"><span class="pre">~/.s3ql</span></tt> or the directory
-specified with <tt class="cmdopt docutils literal"><span class="pre">--homedir</span></tt>. Log files are placed in the same
-directory.</p>
-</div>
<div class="section" id="exit-status">
<h2>Exit Status<a class="headerlink" href="#exit-status" title="Permalink to this headline">¶</a></h2>
<p><strong class="command">mount.s3ql</strong> returns exit code 0 if the operation succeeded and 1 if some
@@ -269,7 +250,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="adm.html" title="The s3qladm command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/pcp.html b/doc/html/man/pcp.html
index 4dff9aa..1f03004 100644
--- a/doc/html/man/pcp.html
+++ b/doc/html/man/pcp.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The pcp command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The pcp command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The expire_backups command" href="expire_backups.html" />
<link rel="prev" title="The fsck.s3ql command" href="fsck.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="fsck.html" title="The fsck.s3ql command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -115,6 +116,9 @@
<strong class="program">sync</strong> processes to copy directory trees in parallel. This is
allows much better copying performance on file system that have
relatively high latency when retrieving individual files like S3QL.</p>
+<p><strong>Note</strong>: Using this program only improves performance when copying
+<em>from</em> an S3QL file system. When copying <em>to</em> an S3QL file system,
+using <strong class="command">pcp</strong> is more likely to <em>decrease</em> performance.</p>
</div>
<div class="section" id="options">
<h2>Options<a class="headerlink" href="#options" title="Permalink to this headline">¶</a></h2>
@@ -170,7 +174,7 @@ error occured.</p>
<li class="right" >
<a href="fsck.html" title="The fsck.s3ql command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/rm.html b/doc/html/man/rm.html
index 2ab13b1..a308d9a 100644
--- a/doc/html/man/rm.html
+++ b/doc/html/man/rm.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The s3qlrm command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The s3qlrm command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The s3qllock command" href="lock.html" />
<link rel="prev" title="The s3qlcp command" href="cp.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="cp.html" title="The s3qlcp command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -118,6 +119,10 @@ you to delete immutable trees (which can be created with
<strong class="program">s3qllock</strong>) as well.</p>
<p>Be warned that there is no additional confirmation. The directory will
be removed entirely and immediately.</p>
+<p><strong class="command">s3qlrm</strong> can only be called by the user that mounted the file system
+and (if the file system was mounted with <tt class="cmdopt docutils literal"><span class="pre">--allow-other</span></tt> or
+<tt class="cmdopt docutils literal"><span class="pre">--allow-root</span></tt>) the root user. This limitation might be
+removed in the future (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=155">issue 155</a>).</p>
</div>
<div class="section" id="options">
<h2>Options<a class="headerlink" href="#options" title="Permalink to this headline">¶</a></h2>
@@ -169,7 +174,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="cp.html" title="The s3qlcp command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/stat.html b/doc/html/man/stat.html
index 9182687..d96a19d 100644
--- a/doc/html/man/stat.html
+++ b/doc/html/man/stat.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The s3qlstat command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The s3qlstat command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The s3qlctrl command" href="ctrl.html" />
<link rel="prev" title="The mount.s3ql command" href="mount.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="mount.html" title="The mount.s3ql command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -168,7 +169,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="mount.html" title="The mount.s3ql command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/man/umount.html b/doc/html/man/umount.html
index 30039fc..24eebf3 100644
--- a/doc/html/man/umount.html
+++ b/doc/html/man/umount.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>The umount.s3ql command &mdash; S3QL 1.0.1 documentation</title>
+ <title>The umount.s3ql command &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="../index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="../index.html" />
<link rel="up" title="Manpages" href="index.html" />
<link rel="next" title="The fsck.s3ql command" href="fsck.html" />
<link rel="prev" title="The s3qllock command" href="lock.html" />
@@ -41,7 +41,7 @@
<li class="right" >
<a href="lock.html" title="The s3qllock command"
accesskey="P">previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Manpages</a> &raquo;</li>
</ul>
</div>
@@ -51,6 +51,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="../backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../adm.html">Managing Buckets</a></li>
@@ -178,7 +179,7 @@ system, conventional locations are <tt class="file docutils literal"><span class
<li class="right" >
<a href="lock.html" title="The s3qllock command"
>previous</a> |</li>
- <li><a href="../index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="../index.html">S3QL 1.2 documentation</a> &raquo;</li>
<li><a href="index.html" >Manpages</a> &raquo;</li>
</ul>
</div>
diff --git a/doc/html/mkfs.html b/doc/html/mkfs.html
index 4acbe34..a627a91 100644
--- a/doc/html/mkfs.html
+++ b/doc/html/mkfs.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>File System Creation &mdash; S3QL 1.0.1 documentation</title>
+ <title>File System Creation &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Managing Buckets" href="adm.html" />
<link rel="prev" title="Storage Backends" href="backends.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="backends.html" title="Storage Backends"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -87,7 +88,7 @@
<div class="section" id="file-system-creation">
<h1>File System Creation<a class="headerlink" href="#file-system-creation" title="Permalink to this headline">¶</a></h1>
-<p>A S3QL file system is created with the <tt class=" docutils literal"><span class="pre">mkfs.s3ql</span></tt> command. It has the
+<p>A S3QL file system is created with the <tt class="docutils literal"><span class="pre">mkfs.s3ql</span></tt> command. It has the
following syntax:</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">mkfs.s3ql </span><span class="ge">[options]</span><span class="l"> </span><span class="nv">&lt;storage url&gt;</span><span class="l"></span>
</pre></div>
@@ -99,16 +100,20 @@ following syntax:</p>
<col class="description" />
<tbody valign="top">
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication
-info. (default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default: <tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to
-get debug messages from all modules. This option can
-be specified multiple times.</td></tr>
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to
+get debug messages from all modules. This option can be
+specified multiple times.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--quiet</span></kbd></td>
<td>be really quiet</td></tr>
@@ -116,19 +121,6 @@ be specified multiple times.</td></tr>
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.</td></tr>
-<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--s3-location <var>&lt;name&gt;</var></span></kbd></td>
-</tr>
-<tr><td>&nbsp;</td><td>Storage location for new S3 buckets. Allowed values:
-<tt class=" docutils literal"><span class="pre">EU</span></tt>, <tt class=" docutils literal"><span class="pre">us-west-1</span></tt>, <tt class=" docutils literal"><span class="pre">ap-southeast-1</span></tt>, or <tt class=" docutils literal"><span class="pre">us-standard</span></tt>.
-(default: EU)</td></tr>
-<tr><td class="option-group">
<kbd><span class="option">-L <var>&lt;name&gt;</var></span></kbd></td>
<td>Filesystem label</td></tr>
<tr><td class="option-group" colspan="2">
@@ -144,10 +136,10 @@ even for unencrypted file systems.</td></tr>
</tbody>
</table>
</div></blockquote>
-<p>Unless you have specified the <tt class=" docutils literal"><span class="pre">--plain</span></tt> option, <tt class=" docutils literal"><span class="pre">mkfs.s3ql</span></tt> will ask you
-to enter an encryption password. If you do not want to enter this
-password every time that you mount the file system, you can store it
-in the <tt class=" docutils literal"><span class="pre">~/.s3ql/authinfo</span></tt> file, see <a class="reference internal" href="mount.html#bucket-pw"><em>Storing Encryption Passwords</em></a>.</p>
+<p>Unless you have specified the <tt class="docutils literal"><span class="pre">--plain</span></tt> option, <tt class="docutils literal"><span class="pre">mkfs.s3ql</span></tt> will ask
+you to enter an encryption password. This password will <em>not</em> be read
+from an authentication file specified with the <tt class="cmdopt docutils literal"><span class="pre">--authfile</span></tt>
+option to prevent accidental creation of an encrypted bucket.</p>
</div>
@@ -165,7 +157,7 @@ in the <tt class=" docutils literal"><span class="pre">~/.s3ql/authinfo</span></
<li class="right" >
<a href="backends.html" title="Storage Backends"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/mount.html b/doc/html/mount.html
index f8de8a4..e0de7dd 100644
--- a/doc/html/mount.html
+++ b/doc/html/mount.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Mounting &mdash; S3QL 1.0.1 documentation</title>
+ <title>Mounting &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Advanced S3QL Features" href="special.html" />
<link rel="prev" title="Managing Buckets" href="adm.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="adm.html" title="Managing Buckets"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,11 +49,11 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="">Mounting</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="#storing-encryption-passwords">Storing Encryption Passwords</a></li>
<li class="toctree-l2"><a class="reference internal" href="#compression-algorithms">Compression Algorithms</a></li>
<li class="toctree-l2"><a class="reference internal" href="#parallel-compression">Parallel Compression</a></li>
<li class="toctree-l2"><a class="reference internal" href="#notes-about-caching">Notes about Caching</a></li>
@@ -94,7 +94,7 @@
<div class="section" id="mounting">
<h1>Mounting<a class="headerlink" href="#mounting" title="Permalink to this headline">¶</a></h1>
-<p>A S3QL file system is mounted with the <tt class=" docutils literal"><span class="pre">mount.s3ql</span></tt> command. It has
+<p>A S3QL file system is mounted with the <tt class="docutils literal"><span class="pre">mount.s3ql</span></tt> command. It has
the following syntax:</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">mount.s3ql </span><span class="ge">[options]</span><span class="l"> </span><span class="nv">&lt;storage url&gt;</span><span class="l"> </span><span class="nv">&lt;mountpoint&gt;</span><span class="l"></span>
</pre></div>
@@ -111,15 +111,26 @@ mounted on one computer at a time.</p>
<col class="option" />
<col class="description" />
<tbody valign="top">
+<tr><td class="option-group">
+<kbd><span class="option">--log <var>&lt;target&gt;</var></span></kbd></td>
+<td>Write logging info into this file. File will be
+rotated when it reaches 1 MB, and at most 5 old log
+files will be kept. Specify <tt class="docutils literal"><span class="pre">none</span></tt> to disable
+logging. Default: <tt class="docutils literal"><span class="pre">~/.s3ql/mount.log</span></tt></td></tr>
+<tr><td class="option-group" colspan="2">
+<kbd><span class="option">--cachedir <var>&lt;path&gt;</var></span></kbd></td>
+</tr>
+<tr><td>&nbsp;</td><td>Store cached data in this directory (default:
+<tt class="docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--homedir <var>&lt;path&gt;</var></span></kbd></td>
+<kbd><span class="option">--authfile <var>&lt;path&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>Directory for log files, cache and authentication
-info. (default: <tt class=" docutils literal"><span class="pre">~/.s3ql)</span></tt></td></tr>
+<tr><td>&nbsp;</td><td>Read authentication credentials from this file
+(default: <tt class="docutils literal"><span class="pre">~/.s3ql/authinfo2)</span></tt></td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--debug <var>&lt;module&gt;</var></span></kbd></td>
</tr>
-<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class=" docutils literal"><span class="pre">all</span></tt> to
+<tr><td>&nbsp;</td><td>activate debugging output from &lt;module&gt;. Use <tt class="docutils literal"><span class="pre">all</span></tt> to
get debug messages from all modules. This option can
be specified multiple times.</td></tr>
<tr><td class="option-group">
@@ -128,13 +139,6 @@ be specified multiple times.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--version</span></kbd></td>
<td>just print program version and exit</td></tr>
-<tr><td class="option-group">
-<kbd><span class="option">--ssl</span></kbd></td>
-<td>Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--cachesize <var>&lt;size&gt;</var></span></kbd></td>
</tr>
@@ -150,20 +154,20 @@ operation.</td></tr>
Each cache entry requires one file descriptor, so if
you increase this number you have to make sure that
your process file descriptor limit (as set with
-<tt class=" docutils literal"><span class="pre">ulimit</span> <span class="pre">-n</span></tt>) is high enough (at least the number of
+<tt class="docutils literal"><span class="pre">ulimit</span> <span class="pre">-n</span></tt>) is high enough (at least the number of
cache entries + 100).</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--allow-other</span></kbd></td>
-<td>Normally, only the user who called <tt class=" docutils literal"><span class="pre">mount.s3ql</span></tt> can
+<td>Normally, only the user who called <tt class="docutils literal"><span class="pre">mount.s3ql</span></tt> can
access the mount point. This user then also has full
access to it, independent of individual file
-permissions. If the <tt class=" docutils literal"><span class="pre">--allow-other</span></tt> option is
+permissions. If the <tt class="docutils literal"><span class="pre">--allow-other</span></tt> option is
specified, other users can access the mount point as
well and individual file permissions are taken into
account for all users.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--allow-root</span></kbd></td>
-<td>Like <tt class=" docutils literal"><span class="pre">--allow-other</span></tt>, but restrict access to the
+<td>Like <tt class="docutils literal"><span class="pre">--allow-other</span></tt>, but restrict access to the
mounting user and the root user.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--fg</span></kbd></td>
@@ -184,45 +188,24 @@ this, then you don&#8217;t need it.</td></tr>
<kbd><span class="option">--compress <var>&lt;name&gt;</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Compression algorithm to use when storing new data.
-Allowed values: <tt class=" docutils literal"><span class="pre">lzma</span></tt>, <tt class=" docutils literal"><span class="pre">bzip2</span></tt>, <tt class=" docutils literal"><span class="pre">zlib</span></tt>, none.
-(default: <tt class=" docutils literal"><span class="pre">lzma</span></tt>)</td></tr>
+Allowed values: <tt class="docutils literal"><span class="pre">lzma</span></tt>, <tt class="docutils literal"><span class="pre">bzip2</span></tt>, <tt class="docutils literal"><span class="pre">zlib</span></tt>, none.
+(default: <tt class="docutils literal"><span class="pre">lzma</span></tt>)</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--metadata-upload-interval <var>&lt;seconds&gt;</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Interval in seconds between complete metadata uploads.
Set to 0 to disable. Default: 24h.</td></tr>
-<tr><td class="option-group" colspan="2">
-<kbd><span class="option">--compression-threads <var>&lt;no&gt;</var></span></kbd></td>
-</tr>
-<tr><td>&nbsp;</td><td>Number of parallel compression and encryption threads
-to use (default: 1).</td></tr>
+<tr><td class="option-group">
+<kbd><span class="option">--threads <var>&lt;no&gt;</var></span></kbd></td>
+<td>Number of parallel upload threads to use (default:
+auto).</td></tr>
+<tr><td class="option-group">
+<kbd><span class="option">--nfs</span></kbd></td>
+<td>Support export of S3QL file systems over NFS (default:
+False)</td></tr>
</tbody>
</table>
</div></blockquote>
-<div class="section" id="storing-encryption-passwords">
-<span id="bucket-pw"></span><h2>Storing Encryption Passwords<a class="headerlink" href="#storing-encryption-passwords" title="Permalink to this headline">¶</a></h2>
-<p>If you are trying to mount an encrypted bucket, <tt class=" docutils literal"><span class="pre">mount.s3ql</span></tt> will first
-try to read the password from the <tt class=" docutils literal"><span class="pre">.s3ql/authinfo</span></tt> file (the same file
-that is used to read the backend authentication data) and prompt the
-user to enter the password only if this fails.</p>
-<p>The <tt class=" docutils literal"><span class="pre">authinfo</span></tt> entries to specify bucket passwords are of the form</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">storage-url </span><span class="nv">&lt;storage-url&gt;</span><span class="l"> password </span><span class="nv">&lt;password&gt;</span><span class="l"></span>
-</pre></div>
-</div>
-<p>So to always use the password <tt class=" docutils literal"><span class="pre">topsecret</span></tt> when mounting <tt class=" docutils literal"><span class="pre">s3://joes_bucket</span></tt>,
-the entry would be</p>
-<div class="highlight-commandline"><div class="highlight"><pre><span class="l">storage-url s3://joes_bucket password topsecret</span>
-</pre></div>
-</div>
-<div class="admonition note">
-<p class="first admonition-title">Note</p>
-<p class="last">If you are using the local backend, the storage url will
-always be converted to an absolute path. So if you are in the
-<tt class=" docutils literal"><span class="pre">/home/john</span></tt> directory and try to mount <tt class=" docutils literal"><span class="pre">local://bucket</span></tt>, the matching
-<tt class=" docutils literal"><span class="pre">authinfo</span></tt> entry has to have a storage url of
-<tt class=" docutils literal"><span class="pre">local:///home/john/bucket</span></tt>.</p>
-</div>
-</div>
<div class="section" id="compression-algorithms">
<h2>Compression Algorithms<a class="headerlink" href="#compression-algorithms" title="Permalink to this headline">¶</a></h2>
<p>S3QL supports three compression algorithms, LZMA, Bzip2 and zlib (with
@@ -236,22 +219,21 @@ compression ratios than zlib.</p>
depends on your network connection speed: the compression algorithm
should be fast enough to saturate your network connection.</p>
<p>To find the optimal algorithm for your system, S3QL ships with a
-program called <tt class=" docutils literal"><span class="pre">benchmark.py</span></tt> in the <tt class=" docutils literal"><span class="pre">contrib</span></tt> directory. You should
+program called <tt class="docutils literal"><span class="pre">benchmark.py</span></tt> in the <tt class="docutils literal"><span class="pre">contrib</span></tt> directory. You should
run this program on a file that has a size that is roughly equal to
the block size of your file system and has similar contents. It will
then determine the compression speeds for the different algorithms and
the upload speeds for the specified backend and recommend the best
algorithm that is fast enough to saturate your network connection.</p>
<p>Obviously you should make sure that there is little other system load
-when you run <tt class=" docutils literal"><span class="pre">benchmark.py</span></tt> (i.e., don&#8217;t compile software or encode
+when you run <tt class="docutils literal"><span class="pre">benchmark.py</span></tt> (i.e., don&#8217;t compile software or encode
videos at the same time).</p>
</div>
<div class="section" id="parallel-compression">
<h2>Parallel Compression<a class="headerlink" href="#parallel-compression" title="Permalink to this headline">¶</a></h2>
<p>If you are running S3QL on a system with multiple cores, you might
-want to set <tt class="docutils literal"><span class="pre">--compression-threads</span></tt> to a value bigger than one. This
-will instruct S3QL to compress and encrypt several blocks at the same
-time.</p>
+want to set the <tt class="docutils literal"><span class="pre">--threads</span></tt> value larger than one. This will
+instruct S3QL to compress and encrypt several blocks at the same time.</p>
<p>If you want to do this in combination with using the LZMA compression
algorithm, you should keep an eye on memory usage though. Every
LZMA compression threads requires about 200 MB of RAM.</p>
@@ -259,7 +241,7 @@ LZMA compression threads requires about 200 MB of RAM.</p>
<p class="first admonition-title">Note</p>
<p class="last">To determine the optimal compression algorithm for your network
connection when using multiple threads, you can pass the
-<tt class="docutils literal"><span class="pre">--compression-threads</span></tt> option to <tt class=" docutils literal"><span class="pre">contrib/benchmark.py</span></tt>.</p>
+<tt class="docutils literal"><span class="pre">--compression-threads</span></tt> option to <tt class="docutils literal"><span class="pre">contrib/benchmark.py</span></tt>.</p>
</div>
</div>
<div class="section" id="notes-about-caching">
@@ -269,17 +251,17 @@ access. The cache is block based, so it is possible that only parts of
a file are in the cache.</p>
<div class="section" id="maximum-number-of-cache-entries">
<h3>Maximum Number of Cache Entries<a class="headerlink" href="#maximum-number-of-cache-entries" title="Permalink to this headline">¶</a></h3>
-<p>The maximum size of the cache can be configured with the <tt class=" docutils literal"><span class="pre">--cachesize</span></tt>
+<p>The maximum size of the cache can be configured with the <tt class="docutils literal"><span class="pre">--cachesize</span></tt>
option. In addition to that, the maximum number of objects in the
-cache is limited by the <tt class=" docutils literal"><span class="pre">--max-cache-entries</span></tt> option, so it is
+cache is limited by the <tt class="docutils literal"><span class="pre">--max-cache-entries</span></tt> option, so it is
possible that the cache does not grow up to the maximum cache size
because the maximum number of cache elements has been reached. The
reason for this limit is that each cache entry requires one open
file descriptor, and Linux distributions usually limit the total
number of file descriptors per process to about a thousand.</p>
-<p>If you specify a value for <tt class=" docutils literal"><span class="pre">--max-cache-entries</span></tt>, you should therefore
+<p>If you specify a value for <tt class="docutils literal"><span class="pre">--max-cache-entries</span></tt>, you should therefore
make sure to also configure your system to increase the maximum number
-of open file handles. This can be done temporarily with the <tt class=" docutils literal"><span class="pre">umask</span> <span class="pre">-n</span></tt>
+of open file handles. This can be done temporarily with the <tt class="docutils literal"><span class="pre">umask</span> <span class="pre">-n</span></tt>
command. The method to permanently change this limit system-wide
depends on your distribution.</p>
</div>
@@ -299,7 +281,7 @@ recently used blocks first.</p>
system startup and shutdown, you should do so with one dedicated S3QL
init script for each S3QL file system.</p>
<p>If your system is using upstart, an appropriate job can be defined
-as follows (and should be placed in <tt class=" docutils literal"><span class="pre">/etc/init/</span></tt>):</p>
+as follows (and should be placed in <tt class="docutils literal"><span class="pre">/etc/init/</span></tt>):</p>
<div class="highlight-commandline"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
@@ -362,16 +344,16 @@ as follows (and should be placed in <tt class=" docutils literal"><span class="p
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>In principle, it is also possible to automatically mount an S3QL
-file system with an appropriate entry in <tt class=" docutils literal"><span class="pre">/etc/fstab</span></tt>. However,
+file system with an appropriate entry in <tt class="docutils literal"><span class="pre">/etc/fstab</span></tt>. However,
this is not recommended for several reasons:</p>
<ul class="last simple">
-<li>file systems mounted in <tt class=" docutils literal"><span class="pre">/etc/fstab</span></tt> will be unmounted with the
-<tt class=" docutils literal"><span class="pre">umount</span></tt> command, so your system will not wait until all data has
+<li>file systems mounted in <tt class="docutils literal"><span class="pre">/etc/fstab</span></tt> will be unmounted with the
+<tt class="docutils literal"><span class="pre">umount</span></tt> command, so your system will not wait until all data has
been uploaded but shutdown (or restart) immediately (this is a
FUSE limitation, see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=159">issue 159</a>).</li>
<li>There is no way to tell the system that mounting S3QL requires a
Python interpreter to be available, so it may attempt to run
-<tt class=" docutils literal"><span class="pre">mount.s3ql</span></tt> before it has mounted the volume containing the
+<tt class="docutils literal"><span class="pre">mount.s3ql</span></tt> before it has mounted the volume containing the
Python interpreter.</li>
<li>There is no standard way to tell the system that internet
connection has to be up before the S3QL file system can be
@@ -396,7 +378,7 @@ mounted.</li>
<li class="right" >
<a href="adm.html" title="Managing Buckets"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/objects.inv b/doc/html/objects.inv
index be510e2..fbb9b14 100644
--- a/doc/html/objects.inv
+++ b/doc/html/objects.inv
Binary files differ
diff --git a/doc/html/resources.html b/doc/html/resources.html
index 37e568d..a90f6db 100644
--- a/doc/html/resources.html
+++ b/doc/html/resources.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Further Resources / Getting Help &mdash; S3QL 1.0.1 documentation</title>
+ <title>Further Resources / Getting Help &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="prev" title="The expire_backups command" href="man/expire_backups.html" />
</head>
<body>
@@ -36,7 +36,7 @@
<li class="right" style="margin-right: 10px">
<a href="man/expire_backups.html" title="The expire_backups command"
accesskey="P">previous</a></li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -45,6 +45,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -107,7 +108,7 @@ can subscribe by sending a mail to
<li class="right" style="margin-right: 10px">
<a href="man/expire_backups.html" title="The expire_backups command"
>previous</a></li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/search.html b/doc/html/search.html
index 30b0686..b1f5b87 100644
--- a/doc/html/search.html
+++ b/doc/html/search.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Search &mdash; S3QL 1.0.1 documentation</title>
+ <title>Search &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -27,7 +27,7 @@
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/searchtools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<script type="text/javascript">
jQuery(function() { Search.loadIndex("searchindex.js"); });
</script>
@@ -38,7 +38,7 @@
<div class="related">
<h3>Navigation</h3>
<ul>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -47,6 +47,7 @@
<ul>
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -102,7 +103,7 @@
<div class="related">
<h3>Navigation</h3>
<ul>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/searchindex.js b/doc/html/searchindex.js
index d4a910d..e9b5e1a 100644
--- a/doc/html/searchindex.js
+++ b/doc/html/searchindex.js
@@ -1 +1 @@
-Search.setIndex({objects:{},terms:{all:[7,16,0,19,21,10,22,20,23,18,24,1,11,26,6,2,4,13,14,25,12],code:[7,16,0,21,10,22,9,24,20,2,3,25,5,19],partial:11,global:11,mnt:[4,13,26],month:[4,24],stumbl:12,per:[26,11],follow:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,18,19,21,22,23,24,25,26],disk:[13,20],intrins:13,locat:[7,16,0,10,22,9,1,11,2,25,5,19,6],whose:11,decid:[13,11],depend:[7,8,21,22,24,2,4,13,19,26],system:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18,19,20,21,22,23,24,25,26],ulimit:[26,19],readabl:20,specif:[0,21,12,25,14,6],send:15,rel:[11,3],init:[26,12],program:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,16,18,19,21,22,23,24,25,26],present:11,under:11,aris:11,sha256:20,neglig:11,worth:12,introduc:[16,6],sourc:[16,20,21,3,4,6],everi:[16,0,26,24,1,11,4,13,6],risk:11,far:13,faq:15,account:[19,11,26],util:[13,18],volum:26,failur:11,veri:[20,11,12],appar:11,tri:[14,11,12],administr:[0,6],level:[25,11,6],did:20,list:[20,21,24,25,11,6,4,14,15],upload:[16,20,10,26,23,11,12,25,13,19,6],"try":[4,24,26,11,21],stderr:26,small:[13,12],refer:[11,17],servic:20,dir:[16,26],pleas:[20,11,15],upper:11,smaller:[13,20],slower:[16,26,6],ifac:26,ten:11,compens:11,sync:3,histor:11,consequ:11,hors:[0,6],design:[0,11,20,6],pass:[26,3],download:[14,21,2],further:[8,14,15],port:[7,19,11,22,2],rath:26,even:[7,0,19,10,22,23,18,1,11,2,14,26,6],what:[4,14,11,25,24],bzip2:[20,19,26],sub:12,descriptor:[26,19],preserv:[4,20],section:11,abl:[0,10,23,11,6,15],invok:[25,14],asia:11,anywai:[7,14,22,18,1,2,19,26],access:[20,19,26,11,14,12],delet:[8,0,9,24,11,6,2,4,14,12],version:[7,16,0,19,21,10,22,9,23,18,24,1,11,20,2,3,25,5,14,26,12],suspect:18,consecut:[7,19,11,22,2],"new":[7,16,0,19,22,26,24,1,11,6,4,14,25,12],net:26,significantli:[4,11,21],ever:11,"public":11,contrast:[0,6],metadata:[8,19,26,2,25,13,14,6],elimin:11,full:[7,16,0,10,22,9,25,20,2,4,5,19,26,6],pacif:11,gener:[0,21,24,12,4,14,6],never:[7,14,22,18,1,2,19,26],privat:11,here:[25,17],satisfi:21,slow:[11,12],modif:12,address:12,path:[7,19,21,22,18,1,11,2,14,26],becom:[4,24,13,11],modifi:[16,20,6],sinc:[4,24,26,11,12],valu:[1,22,19,26,12],wait:[10,26,23,11,13,12],dry:24,convert:26,joes_bucket:26,checksum:20,current:[4,14,11,24,12],host:[7,19,11,22,2],loos:11,amount:[25,13,11,20],throughput:4,shrink:20,action:[25,14,6,2],chang:[16,8,0,26,20,11,6,2,25,14,12],magnitud:[16,6],chanc:[11,12],control:6,configur:[8,26,6],retriev:[13,19,26,3],appli:11,modul:[7,14,21,22,18,1,2,25,19,26],apw:21,filenam:13,unix:[13,20,12],leav:12,visibl:11,instal:[7,8,0,19,17,21,16,10,22,9,25,2,4,5,14],total:[26,11,6,12],establish:11,kei:[20,11],from:[7,16,0,19,21,22,20,18,24,1,11,2,4,13,14,26,6],describ:[],would:[16,26,24,11,4,6],apswvers:21,upgrad:[8,14,21,2],few:[4,11,21],concret:11,call:[16,0,26,24,11,4,5,13,19,6],usr:[7,16,0,10,22,9,25,2,4,5,19],recommend:[4,26,11,12],taken:[16,20,26,11,19,6],suppos:11,type:11,until:[10,26,6,12,23],more:[20,21,11,12,25,6],sort:[4,24],desir:25,st_nlink:12,comparison:[],peopl:11,hundr:11,relat:11,benchmark:[4,8,26],deflat:20,notic:[13,11],enhanc:4,warn:[25,14,9,11,6],sqlitelibvers:21,depth:[],sens:13,known:[8,20,12],actual:[14,13,11,21,12],destin:[16,6,3],unpack:21,cach:[7,8,20,19,16,22,18,1,12,2,25,14,26,6],must:[7,16,19,22,11,2,14,6],worst:[0,6],none:[19,11,26],word:11,restor:[16,8,13,14,6],setup:21,work:[20,21,23,24,11,4,13,12],conceptu:20,remain:[26,11],wors:11,obvious:26,can:[0,1,2,4,5,6,7,9,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26],root:[16,10,26,23,5,19,6],fetch:[16,6],overrid:11,prompt:[7,26,18],tar:13,give:[4,24,14],process:[10,26,23,11,3,4,13,19,12],lock:[0,6],sudo:21,share:[7,16,0,10,22,9,25,2,4,5,19],accept:[7,16,0,19,10,22,9,23,18,24,1,2,3,25,5,14,26],high:[20,19,26,3],critic:20,minimum:[13,20],want:[21,10,26,23,24,1,11,4,13,15],onlin:[0,20,6],acl:20,occur:[7,16,0,10,22,9,24,11,2,3,25,5,19],alwai:[16,26,24,11,12,4,6],algorithm:[4,8,20,19,26],end:[26,14],manipul:20,thing:[14,2],rather:[24,11,12,4,13,6],anoth:[11,12],ordinari:[16,10,23,6],write:[7,8,20,21,16,22,11,12,2,13,19,26,6],how:[4,24,13,11],env:26,instead:[16,10,23,11,12,6],simpl:[4,24,6],confid:[],product:[],resourc:[8,14,15],haven:11,max:[26,19],earlier:[11,21,12],pyliblzma:21,badli:[14,2],wrong:[14,12,2],compat:12,endang:20,mai:[16,0,19,21,26,15,20,23,25,11,6,2,4,13,14,12],multipl:[7,14,22,18,1,2,19,26],redund:[4,24,20,11],autom:[4,20],data:[7,16,20,19,10,22,26,23,18,1,11,6,2,4,13,14,25,12],grow:[20,26],physic:[4,16,0,14,6],man:17,handl:[16,20,26,24,11,12,4,13,6],"short":[0,11,6],attempt:[7,26,11,12],practic:[13,11],third:13,read:[7,16,20,22,11,12,2,13,19,26,6],minim:20,favorit:11,apsw:21,element:26,issu:[16,8,20,21,12,11,6,5,26,15],inform:[26,24,11,25,19,6],maintain:26,combin:[26,11],allow:[16,0,19,22,9,24,1,11,26,3,4,5,13,14,25,6],enter:[1,26],exclus:13,pycryptopp:21,order:[16,6],origin:[],help:[8,21,15,25,12,2,4,14,6],over:[16,20,13,6,12],move:[14,20,11],soon:11,topsecret:26,report:[20,11,21,15],affect:[16,26,11,6,12],effici:12,still:[16,20,10,26,23,11,6],dynam:20,fix:[0,24,11,12,4,6],inadvert:[16,6],better:[26,11,21,3],window:11,requir:[7,20,21,26,18,24,25,4,19,12],persist:11,mail:[20,14,21,15],main:[9,6],might:[16,12,18,6,5,26,15],split:20,them:[16,10,23,11,14,6],good:[0,24,12,4,13,6],synopsi:[7,16,0,10,22,9,24,2,3,25,5,19],thei:[16,0,17,20,24,11,4,13,6],python:[26,21],promin:12,safe:[16,13,11,6],dai:[4,24],initi:11,dat:[4,24],therebi:11,interrupt:14,now:[0,21,24,11,4,6],bigger:26,choic:21,term:[4,24],document:[7,16,0,10,22,9,2,25,5,19,6],somewher:[7,16,0,10,22,9,2,25,5,19,6],name:[7,16,22,24,1,11,2,4,13,19,26,6],hardlink:[16,20,6],joesdomain:11,authent:[7,19,22,18,1,11,2,14,26],achiev:[16,20,26,6],mode:[26,19],each:[4,24,26,19],debug:[7,16,0,19,10,22,9,23,18,24,1,2,3,25,5,14,26],found:[13,11],beneath:6,updat:[4,20,11,12],side:11,mean:[20,11,12],compil:26,s3ql:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26],mkfifo:26,chunk:12,hard:[13,20],idea:[4,24,0,6,12],procedur:[16,6],realli:[7,16,0,19,10,22,9,23,18,24,1,11,2,3,25,5,13,14,26,12],ensur:[4,24,13,20],meta:[25,13,6],expect:[26,11],year:[4,24,20,11],s3_copi:[4,8],happen:[0,11,6],special:[16,6],out:[20,11,12],lzma:[20,19,21,26],ftp:11,shown:11,network:[16,13,26,20,6],space:[16,20,6,12],miss:[24,11],newli:12,content:[16,20,26,0,11,2,6],suitabl:[20,21],rational:0,internet:[20,26],print:[7,16,0,19,21,10,22,9,23,18,24,1,2,3,25,5,14,26],eth0:26,bla:26,correct:[4,24],statist:[5,6,8],foreground:[26,19],shut:[4,10,23,12],after:[16,0,20,11,13,14,6],insid:16,advanc:[8,6],migrat:4,documents_janurai:[16,6],million:[11,12],situat:[13,11,12],differ:[16,20,21,26,25,11,2,4,13,14,6],free:[20,11],standard:[20,22,24,1,11,4,26],inod:12,reason:[9,26,23,18,24,11,12,4,13,6],base:26,theori:[11,12],ask:[1,11,21,14,12],org:26,"byte":[],likelihood:11,afterward:13,bash:13,care:[16,20,11,6,12],wai:[20,26,0,11,12,13,6],thread:[26,19],befor:[16,20,10,26,23,11,12,6],guarante:[4,8,11,24],could:[16,6],refus:[4,24,11],keep:[0,26,24,11,4,6],recov:[0,6],turn:[4,24,26,12],fuse:[26,21,12],place:[7,26,11,2,19,22],perman:[8,0,26,11,4,13,6],principl:26,confus:12,neglect:11,first:[20,21,26,11,13,14,12],oper:[7,16,0,19,10,22,9,24,11,20,2,3,25,5,14,26],softwar:26,rang:[4,24],malici:[0,6],carri:20,onc:[0,10,26,20,23,12,13,19,6],independ:[19,11,26,20],number:[20,21,26,24,11,12,3,4,13,19,6],capac:20,restrict:[7,22,11,2,25,19,26],date:[4,24],instruct:[26,21],alreadi:[7,16,19,21,22,18,1,11,12,2,14,26,6],done:[26,11],least:[21,26,24,11,4,19],llfuse:21,indistinguish:20,open:[10,26,23,12],size:[20,22,1,12,25,13,19,26,6],given:12,"long":[4,24,13,11,12],convent:[7,16,0,10,22,9,2,25,5,19,12],script:[4,13,26,12],unknown:[20,11],interact:2,s3qllock:[8,0,17,9,4,13,6],sometim:14,wrapper:[4,3],parallel:[8,26,3,4,13,19],checkpoint:11,attack:[0,6],necessarili:11,demonstr:4,s3qlrm:[8,0,17,9,24,4,6],termin:[10,11,23,12],john:26,"final":6,store:[7,8,20,22,24,1,11,2,4,13,19,26,12],includ:[21,20,11,17,12],luckili:12,option:[0,1,2,3,4,5,6,7,9,10,12,13,14,16,18,19,21,22,23,24,25,26],especi:[13,20],shelf:[0,6],flushcach:[25,6],copi:[16,8,20,11,12,3,4,13,14,6],specifi:[7,19,22,26,18,24,1,11,2,4,14,25,12],part:[4,24,20,26,3],mostli:11,consult:[],exactli:[4,16,11,24,6],than:[16,9,26,24,11,12,4,13,6],wide:[26,21],target:[16,6],whenev:[13,21,26],prevent:[0,6],remov:[16,8,0,21,26,9,24,6,4,5,12],older:14,tree:[16,8,0,9,3,4,20,6],second:[26,24,11,4,13,19],structur:[20,14],matter:[25,13,14],optimum:13,num:[26,19],friend:13,video:26,minut:[4,24,11],pre:26,unencrypt:[7,19,22,18,1,11,2,14,26],sai:[16,6],comput:[7,20,22,24,11,2,4,19,26],entail:11,plaintext:[7,14,22,18,1,2,19,26],explicit:13,ram:26,mind:11,argument:[25,14,2],"13th":20,packag:[4,21],expir:26,increment:[],close:[10,23],need:[16,19,21,26,24,11,4,14,6],seem:[13,11],exclud:[0,6],paramiko:21,caus:11,equival:[4,24],irrepar:14,destroi:[0,6],moreov:6,blindli:12,atim:12,accuraci:[4,24],note:[7,8,20,19,21,16,10,22,23,24,11,6,2,4,14,26,12],also:[7,16,0,17,21,10,22,9,23,24,11,20,2,3,25,5,13,19,26,6],builtin:21,without:[7,16,0,20,18,11,12,13,14,6],take:[16,20,11,12,2,25,13,14,6],which:[16,9,21,20,24,11,12,4,6],discrep:[],transmit:[7,14,22,18,1,2,19,26],environ:21,uplink:4,singl:[13,19,26],mktemp:26,begin:11,sure:[19,21,26,11,13,14,12],unless:[1,0,6],distribut:[4,26,21],plenti:[],normal:[26,19],buffer:12,previou:[4,14],compress:[8,20,26,4,19,6],most:[16,24,11,12,4,14,6],beta:20,detect:[20,11],rigor:[4,24],plan:11,homepag:[7,16,0,10,22,9,2,25,5,19],"class":[7,16,0,10,22,9,24,25,2,3,4,5,19],relatim:12,simplic:20,renam:20,inconveni:20,url:[7,19,22,18,1,11,2,14,26],doc:[7,16,0,10,22,9,25,2,4,5,19],clear:[16,6],later:[16,0,21,11,12,6],cover:13,drive:[0,6],destruct:[0,6],doe:[20,11,26,23,12],declar:20,snapshot:[16,8,20,25,6],runtim:[8,6],determin:[26,11],occasion:11,left:12,hmac:20,gracefulli:[4,24],myawssecretaccesskei:11,show:24,carefulli:11,random:11,syntax:[26,18,1,25,14,6],permiss:[4,20,19,26],bucketnam:[7,19,11,22,2],newest:[13,2],find:[0,11,26,6,12],redirect:26,absolut:[7,22,24,11,2,4,19,26],onli:[16,0,19,21,10,26,20,23,24,25,11,6,4,5,13,14,12],explicitli:12,ratio:26,just:[7,16,0,19,10,22,9,23,18,24,1,2,3,25,5,13,14,26,6],transact:20,fstab:[26,12],activ:[7,16,0,19,10,22,9,23,18,24,1,2,3,25,5,14,26,6],enough:[13,19,26,12],start:[16,21,26,3,4,13,6],peculiar:11,authinfo:[7,8,22,1,11,2,19,26],latenc:[13,20,3],tape:[0,6],factor:11,folder:[16,21,6],local:[7,8,0,21,16,10,22,9,11,20,2,25,5,13,19,26,6],defin:[4,24,26,11],contribut:[4,8],variou:[25,14,2],get:[7,8,0,19,21,22,26,18,24,1,11,15,2,4,14,25,6],googlegroup:15,nasti:[0,6],stop:26,secondari:[],regularli:[16,6],ssl:[7,14,22,18,1,2,19,26],s3rr:11,cannot:[4,24],ssh:11,increas:[13,19,26],reconstruct:[4,24,11],restart:[26,12],myawsaccesskeyid:11,reveal:20,enabl:[7,14,22,18,1,2,19,26],dramat:13,"19283712_yourname_s3ql":11,method:[26,21],provid:[20,11,21,26,12],bad:13,common:[16,0,6],contain:[21,26,24,11,2,4,14],userspac:[],nowher:[],where:[25,14,11,12,2],caches:[25,26,19,6],wiki:[21,15],kernel:21,set:[4,13,19,26],noleaf:12,proce:21,startup:26,displai:12,see:[7,16,0,19,10,22,9,24,1,11,26,6,2,3,4,5,13,14,25,12],temporarili:26,s3qlcp:[4,8,16,17,6],corrupt:11,disadvantag:[16,6],"__version__":21,becaus:[7,16,20,19,22,18,1,11,2,4,14,26,6],whatsoev:[0,20,6],best:[13,21,26],concern:11,infinit:20,awar:11,statu:[7,8,0,16,10,22,9,24,11,20,2,3,25,5,19,12],said:11,extend:20,correctli:[12,18],hopefulli:12,databas:20,boundari:[4,24],label:[1,22],favor:20,state:[4,24,20,12],between:[4,24,19,11,26],"import":[4,11,21],neither:[0,6],param:14,across:20,attribut:20,amazon:[7,8,20,22,11,2,4,19,12],august:11,manpag:[8,17],weak:11,southeast:[1,22],job:[4,26],entir:[20,0,9,6],joe:[4,11],expire_backup:[4,8,17,24],solv:13,come:[4,13,11,20],local0:26,addit:[16,9,26,20,11,2,25,14,6],both:[16,20,11,21,6],protect:[0,20,6],accident:[0,6,12],irregular:[4,24],extens:[20,11],someth:[16,6],howev:[16,0,21,26,20,24,11,12,4,13,6],alon:11,lazi:[10,23],against:[0,20,6],etc:[16,26,6,12],inconsist:12,exec:26,freeli:26,login:11,com:[7,16,0,21,10,22,9,24,11,2,3,25,5,19,15],pcp:[4,8,13,17,3],load:26,simpli:[0,6],figur:11,inspir:[16,6],period:[0,11,6],insist:13,written:[19,11,26,20],littl:[26,21],shutdown:26,linux:[26,21],averag:11,typic:[16,6],guid:[8,17],assum:26,duplic:[4,16,20,11,6],quit:21,worthless:[0,6],strong:[7,16,0,10,22,9,24,11,2,3,25,5,19],west:[1,22,11],devic:26,three:[7,21,22,11,2,19,26],been:[16,0,10,26,20,23,18,24,11,12,4,13,14,6],mark:[7,18],compon:20,secret:11,much:3,interpret:26,interest:6,subscrib:15,monthli:[16,6],immedi:[9,10,26,23,11,6],strategi:[4,24],infect:[0,6],upstart:[4,26,19],great:[0,6],ani:[7,16,0,21,22,20,24,1,11,15,4,6],zero:13,understand:[26,19],els:14,s3qlstat:[5,17,6,12,8],those:11,"case":[16,0,21,20,24,11,4,13,14,6],replic:[4,16,20,6],trojan:[0,6],ident:[16,20,11,6],look:13,gnu:12,solid:20,plain:[1,22],mount:[16,8,0,19,17,10,26,23,1,11,6,2,4,5,13,14,25,12],zlib:[26,19],batch:[7,26,18],trick:[8,13],documents_februari:[16,6],weren:15,abov:[4,24,11],error:[7,8,0,16,10,22,9,18,24,11,20,2,3,25,5,19,12],invoc:[4,24],ahax:3,region:[4,11],jibbadup:11,bin:[13,21],argpars:21,have:[7,16,0,19,21,10,22,15,20,23,24,1,11,6,2,3,4,13,14,26,12],advantag:[4,24,0,6],stdout:26,almost:13,therefor:[4,26,11,12],remount:11,worri:[11,12],quiet:[7,16,0,19,10,22,9,23,18,24,1,2,3,25,5,14,26],exit:[7,16,0,19,10,22,9,23,18,24,1,2,3,25,5,14,26],conf:[4,8],incom:11,revis:[14,2],dedic:[26,12],sever:[20,21,26,11,3,4,13,19],tamper:[4,24],unmount:[8,10,26,23,18,11,4,13,12],develop:[8,20],author:[26,12],perform:[8,20,21,26,25,11,2,3,4,13,14,12],make:[16,0,19,21,26,20,24,11,12,4,13,14,6],same:[7,16,19,22,24,11,2,4,13,14,26,6],"while":[0,21,26,20,23,11,25,6],dest:16,instanc:13,unexpectedli:12,nikolau:26,pai:11,eventu:11,infer:[4,24],complet:[16,21,26,24,25,11,4,19,6],week:[4,24],archiv:[7,20,22,11,2,19],lie:16,optim:[4,26],keyr:11,confidenti:11,upon:12,effect:[16,20,11,6],solut:4,remot:[7,20,19,22,18,1,11,2,14,26],rais:[26,19],temporari:11,user:[7,8,0,17,21,16,10,26,20,23,18,11,5,19,6],mani:[4,24,20,11],extern:[0,6],encrypt:[7,8,20,19,22,18,1,11,2,14,26],tradition:[0,6],recent:[4,24,26,14,12],appropri:26,eleg:20,nevertheless:[4,11],entri:[19,11,26,12],irrelev:12,well:[0,19,26,9,24,11,4,20,14,6],object:[19,11,26],exampl:[20,24,11,12,4,13,6],command:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,16,17,18,19,21,22,23,24,25,26],thi:[0,1,2,3,4,5,6,7,11,12,13,14,15,16,18,19,20,21,22,23,24,25,26],filesystem:[1,22,19,26,18],gzip:20,fail:[26,11,21,12],spend:13,usual:[16,26,11,6],compromis:13,identifi:11,execut:[4,14,21,6],less:[11,12],conform:[7,19,11,22,2],when:[7,0,19,22,20,18,24,1,11,6,2,3,4,13,14,26,12],obtain:11,rest:13,detail:[4,13,11],bandwidth:[4,20],touch:[16,6],passphras:[7,8,14,22,2,19],roughli:26,speed:[4,26],yet:[16,11,6,12],damag:[4,14,11,24],viru:[0,6],detach:[10,23],homedir:[7,14,22,18,1,2,19,26],easi:13,hint:[25,14,2],point:[10,19,11,26,20],had:[4,16,24,6],s3qladm:[8,14,17,2],theoret:[13,11],add:[0,11,6],other:[16,20,21,26,11,6,5,19,15],nor:[0,6],versa:[16,6],runlevel:26,logger:26,subsequ:[4,24,11],match:26,futur:[16,5,6,12],earli:12,applic:12,transpar:20,webpag:11,big:[13,11,12],ctime:12,know:[16,24,11,12,4,6],background:[10,23],amp:13,bit:[20,11],password:[1,26,11,8],recurs:[4,8,9,6],you:[0,1,2,4,6,7,9,10,11,12,13,14,15,16,18,19,20,21,22,23,24,26],like:[16,0,26,20,11,12,3,19,6],loss:11,daemon:[26,19],lost:11,success:11,arbitrari:20,should:[7,16,0,19,21,10,22,9,23,18,11,6,2,25,5,13,14,26,12],anyth:[],manual:[10,21,23,15],resolv:[11,15],noth:[0,6],necessari:14,either:[11,12,2,25,14,6],furthermor:20,output:[7,16,0,19,10,22,9,23,18,24,1,2,3,25,5,14,26],page:17,two:[16,20,14,21,6],yyyi:[4,24],imagin:[4,24,0,6],right:13,old:[0,21,24,11,4,6],often:[13,11],deal:11,interv:[4,24,13,19,26],creation:[1,11,8],some:[7,16,0,21,10,22,9,23,18,24,25,11,20,2,3,4,5,19,12],umount:[8,17,10,26,23,12],self:21,strongest:11,"export":12,flush:[16,26,6],home:[4,13,26],server:[7,20,19,22,18,1,2,14,26,12],librari:[21,12],"24h":[26,19],basic:11,rsync:[16,0,12,3,4,13,6],confirm:[9,6],"function":14,avoid:[4,24],though:[26,11],definit:[4,24],februari:[16,6],protocol:11,backward:[],usernam:11,equal:26,larg:[16,13,6,12],slash:[7,19,11,22,2],cif:26,backend:[7,8,21,16,10,22,23,11,6,2,25,13,19,26,12],blocksiz:[1,22,19,26],machin:11,core:[4,26],plu:11,who:[10,26,19,23],run:[0,19,21,26,18,24,25,11,6,2,4,14,12],power:4,reach:26,view:17,usag:[4,16,26,24,6],symlink:20,speak:26,step:4,unreason:12,although:[20,9,6,12],januari:[16,6],immut:[8,0,9,4,20,13,6],impos:11,stage:[],continu:23,src:[16,6],about:[8,20,26,11,12,5,13,6],rare:11,memori:26,http:[7,16,0,21,10,22,9,24,11,2,3,25,5,19],storebackup:[16,6],unfortun:[13,11,12],her:[0,6],messag:[7,14,22,18,1,2,19,26],commit:[11,23,12],backup:[16,8,0,26,20,24,11,2,4,13,14,6],disabl:[26,19],block:[20,10,22,23,1,11,12,25,26,6],repair:7,client:12,real:11,encount:[15,12],within:6,encod:26,automat:[4,8,26,11,14],due:[11,6,12],down:[4,10,23,12],ahv:13,contrib:[4,13,26],insuffici:12,storag:[7,8,0,19,16,10,22,20,23,18,1,11,6,2,4,13,14,26,12],your:[7,16,0,17,21,10,22,9,24,25,11,2,4,5,13,19,26,6],durabl:11,manag:[8,14],mkf:[7,1,22,17,8],fast:[8,20,11,26,6],fusermount:[10,23,12],prepar:25,suffici:11,transfer:[4,13,11,20],support:[7,16,20,22,11,12,2,19,26,6],question:15,s3_backup:[4,8],overwrit:[1,22,11],avail:[0,26,24,12,4,14,6],intellig:[4,24,20],reli:[11,12],trigger:[11,6],low:[20,11],lot:[16,13,6],"var":[7,19,11,22,2],succeed:[7,16,0,10,22,9,24,2,3,25,5,19],individu:[26,19,3],lowest:11,properli:4,tracker:[20,21,15],form:[7,22,24,11,2,4,19,26],offer:[16,11,6],forc:[7,1,22,18],regard:[16,6],back:[0,6],sigstop:[26,19],satur:26,measur:4,newer:[11,21],don:[26,19,12],line:[25,20,11],bug:[20,21,15,12],longer:[16,0,20,23,24,11,12,4,14,6],info:[7,14,22,18,1,2,25,19,26],made:[16,20,0,11,13,6],input:[7,18],consist:[8,11],possibl:[20,21,26,0,11,12,14,6],"default":[7,19,22,18,24,1,11,2,3,4,14,26],bucket:[7,8,19,22,1,11,2,4,14,26,12],maximum:[1,22,19,26],tell:[4,24,26,11],asynchron:20,below:6,limit:[16,26,11,12,5,19,6],unnot:11,problem:[7,16,0,15,20,24,11,12,4,13,6],similar:[26,11],reliabl:[8,11],connect:[7,20,19,22,18,1,11,2,13,14,26],featur:[16,8,0,20,11,6],creat:[16,0,19,22,9,24,1,11,6,4,20,14,26,12],certain:11,dure:[14,19,11,26],day_hour:[4,24],stabl:20,s3qlctrl:[25,8,13,17,6],strongli:[4,24],workaround:13,exist:[16,0,22,9,24,1,11,4,6],file:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18,19,20,22,23,24,25,26],mtime:12,request:11,ship:[24,26,21,3],improv:[8,13],check:[7,8,21,26,18,11,14,12],probabl:[20,24,4,13,14,6],otherwis:[26,19],again:[0,11,6,12,23],readi:[],rsync_arg:13,umask:26,googl:[7,16,0,21,10,22,9,24,2,3,25,5,19],tip:[8,13],exception:11,field:11,valid:6,creep:20,rememb:11,test:[20,11,21],presum:11,thousand:[26,11,12],rid:[0,6],particular:11,variabl:21,intend:[],clean:[7,18],fulli:12,mountpoint:[26,23,25,5,19,6],"return":[7,16,0,10,22,9,23,24,11,2,3,25,5,19,12],fsck:[7,8,17,26,18,11,14],briefli:[],releas:20,track:[4,24],log:[7,19,22,18,1,11,2,25,14,26,6],consid:[11,15],sql:23,log_fifo:26,dd_hh:[4,24],pool:11,stai:[19,11,26],reduc:[4,11],infrequ:11,faster:[9,6],pycrypto:21,vice:[16,6],directori:[7,16,0,19,21,10,22,9,18,24,1,11,6,2,3,4,20,13,14,26,12],cycl:[4,24],descript:[7,16,0,10,22,9,24,2,3,25,5,13,19,26],save:[24,13,11,20],rule:13,sftp:[7,8,20,21,22,11,2,19,12],itself:[11,21,12],ignor:11,time:[7,0,19,22,20,18,24,1,11,6,2,4,13,14,26,12],profil:[26,19],daili:[]},objtypes:{},titles:["The <strong class=\"program\">s3qllock</strong> command","File System Creation","The <strong class=\"program\">s3qladm</strong> command","The <strong class=\"program\">pcp</strong> command","Contributed Programs","The <strong class=\"program\">s3qlstat</strong> command","Advanced S3QL Features","The <strong class=\"program\">fsck.s3ql</strong> command","S3QL User&#8217;s Guide","The <strong class=\"program\">s3qlrm</strong> command","The <strong class=\"program\">umount.s3ql</strong> command","Storage Backends","Known Issues","Tips &amp; Tricks","Managing Buckets","Further Resources / Getting Help","The <strong class=\"program\">s3qlcp</strong> command","Manpages","Checking for Errors","The <strong class=\"program\">mount.s3ql</strong> command","About S3QL","Installation","The <strong class=\"program\">mkfs.s3ql</strong> command","Unmounting","The <strong class=\"program\">expire_backups</strong> command","The <strong class=\"program\">s3qlctrl</strong> command","Mounting"],objnames:{},filenames:["man/lock","mkfs","man/adm","man/pcp","contrib","man/stat","special","man/fsck","index","man/rm","man/umount","backends","issues","tips","adm","resources","man/cp","man/index","fsck","man/mount","about","installation","man/mkfs","umount","man/expire_backups","man/ctrl","mount"]}) \ No newline at end of file
+Search.setIndex({objects:{},terms:{suffici:1,all:[0,1,2,3,5,7,8,11,12,13,14,15,17,19,20,21,22,23,24,25,26,27],code:[8,17,0,22,11,23,10,25,21,3,4,26,6,20],partial:12,global:[],mnt:[5,14,27],month:[5,25],prefix:[12,1],stumbl:13,notquitesecret:1,follow:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,19,20,22,23,24,25,26,27],disk:[14,21],whose:12,decid:[14,1],middl:13,depend:[8,9,22,23,1,25,3,5,14,20,27],million:13,ulimit:[27,20],readabl:21,send:16,rel:[12,4],init:[27,13],program:[0,2,3,4,5,6,7,8,9,10,11,13,15,17,19,20,22,23,24,25,26,27],those:12,under:[12,22],aris:12,sha256:21,neglig:[],worth:13,introduc:[17,7],sourc:[17,21,22,4,5,7],everi:[17,0,27,1,25,12,5,14,7],risk:[],mkf:[8,2,23,18,9],far:14,faq:16,account:[20,12,27],util:[14,19],pycryptopp:22,failur:1,veri:[21,12,13],affect:[17,27,7,13,1],tri:[15,13],administr:[0,7],level:[26,7],did:21,who:[11,27,20,24],list:[21,22,25,26,12,7,5,15,16],upload:[21,11,27,24,26,12,13,5,14,20,7],"try":[5,25,12,22,1],larg:[17,14,7,13],stderr:27,small:[14,13],blocksiz:[2,23,20,27],mount:[17,9,0,20,18,11,27,10,24,1,26,12,7,3,5,6,14,15,13],dir:[17,27],pleas:[21,16],upper:12,smaller:[14,21,13],slower:[17,27,7],ifac:27,ten:12,whitespac:[],compens:1,sync:4,sign:12,consequ:1,hors:[0,7],design:[0,21,7,1],pass:[27,4],download:[15,22,3],further:[9,15,16],correspond:12,port:12,rath:27,even:[8,0,11,24,1,12,19,7],what:[5,25,15,26,1],bzip2:[21,20,27],sub:13,entir:[21,0,10,7],descriptor:[27,20],section:[12,1],abl:[0,11,24,1,12,7,16],weren:16,asia:[],find:[0,12,27,7,13],access:[21,20,27,1,12,15,13],delet:[9,0,10,25,12,7,3,5,15,13],version:[8,17,0,20,22,11,23,10,24,19,25,2,12,21,3,4,26,6,15,27,13],suspect:19,consecut:12,"new":[8,0,20,23,1,25,26,12,7,5,15,27,13],net:27,ever:1,"public":[],contrast:[0,7],metadata:[9,20,27,3,26,14,15,7],elimin:12,full:[8,17,0,11,23,10,26,21,3,5,6,20,27,7],themselv:1,absolut:[5,25,12],pacif:[],gener:[9,0,22,1,25,12,7,5,15,13],never:[],privat:[],here:[26,12,18],satisfi:22,explicitli:13,modif:13,address:13,path:[8,20,22,23,19,2,12,3,15,27],becom:[5,25,14,12,1],modifi:[17,21,7],sinc:[27,1,25,12,5,13],valu:[27,20,13,1],wait:[11,14,27,24,13],dry:25,convert:12,joes_bucket:[],checksum:21,larger:27,step:5,amount:[26,14,12,21],throughput:[],action:[26,15,7,3],implement:1,magnitud:[17,7],chanc:13,control:7,fstab:[27,13],appli:[14,12,1],modul:[8,20,1,23,19,2,3,26,22,15,27],apw:22,filenam:14,unix:[14,21,13],visibl:12,instal:[8,9,0,20,18,22,17,11,23,10,26,3,5,6,15],total:[27,7,13],establish:1,from:[8,17,0,20,22,19,23,21,1,25,2,12,3,4,5,14,15,27,7],describ:[12,1],would:[17,1,25,12,5,7],apswvers:22,upgrad:[9,15,22,3],few:[5,12,22],concret:1,call:[17,0,27,10,1,25,26,12,5,6,14,20,7],usr:[8,17,0,11,23,10,26,3,5,6,20],recommend:[5,27,12,13,1],taken:[17,21,20,27,7],tape:[0,7],bill:1,type:[],until:[11,27,7,13,24],more:[21,22,1,13,4,26,7],sort:[5,25],desir:26,st_nlink:13,src:[17,7],peopl:[],hundr:[],relat:12,benchmark:[5,9,27],"19283712_yourname_s3ql":[],notic:[14,12],enhanc:5,warn:[26,10,15,7,1],sqlitelibvers:22,sens:14,known:[9,21,13],rare:[],hold:[12,1],unpack:22,cach:[8,9,21,20,23,19,2,13,3,26,15,27,7],must:[17,15,12,7,3],worst:[0,7],none:[8,20,1,27,19,3,15],word:[],sometim:15,restor:[17,9,14,15,7],dest:17,setup:22,work:[21,22,24,25,5,14,13],uniqu:1,conceptu:21,remain:27,wors:12,obvious:27,can:[0,1,2,3,5,6,7,8,10,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27],mktemp:27,about:[9,21,27,13,6,14,7],root:[17,0,11,27,10,24,26,6,20,7],fetch:[17,7],overrid:12,prompt:[8,19],tar:14,give:[5,25,14,15],process:[11,27,24,12,4,5,14,20,13],lock:[0,7],sudo:22,share:[8,17,0,11,23,10,26,3,5,6,20],accept:[8,17,0,20,19,11,23,10,24,1,25,2,3,4,26,6,15,27],high:[21,20,27,4],critic:21,minimum:[14,21],want:[11,27,24,25,5,14,16],onlin:[0,12,21,7],unfortun:[14,12,13],occur:[8,17,0,11,23,10,1,25,3,4,26,6,20],ratio:27,alwai:[17,27,25,13,5,7],end:[27,15],turn:[5,25,27,13],rather:[25,12,13,5,14,7],anoth:[13,1],ordinari:[17,11,24,7],write:[8,9,21,20,22,17,27,19,12,7,3,5,14,15,13],how:[5,25,14],manpag:[9,18],env:27,webpag:12,verifi:[12,13],simpl:[5,25,7],updat:[5,21,12,13],product:[],resourc:[9,15,16],max:[27,20],earlier:[22,13,1],pyliblzma:22,badli:[15,3],wrong:[15,13,3],endang:21,mai:[17,0,20,22,27,16,21,24,1,26,12,7,3,5,14,15,13],multipl:[8,15,23,19,2,3,20,27],redund:[5,25,21,12],secondari:[],data:[8,17,21,20,19,11,23,27,24,1,2,12,7,3,5,14,15,26,13],grow:[21,27],physic:[5,17,0,15,7],man:[18,13],indistinguish:21,"short":[0,12,7],attempt:[8,27,13,1],practic:[14,12],third:14,read:[8,17,21,20,19,23,1,2,12,7,3,14,15,27,13],neitheristhi:1,author:[27,13],favorit:[],apsw:22,element:27,issu:[17,9,0,22,27,16,10,21,12,7,26,6,13],inform:[9,27,1,25,26,20,7],maintain:27,combin:[14,12,27,1],allow:[17,0,20,27,10,25,26,12,4,5,6,14,15,7],enter:[2,23],exclus:14,volum:27,order:[17,7,1],talk:12,oper:[8,17,0,20,11,23,10,1,25,21,3,4,26,6,15,27],help:[9,22,16,26,13,3,5,15,7],over:[17,21,27,12,13,14,20,7],move:[21,15,1],soon:[],topsecret:[],increas:[14,20,27],appar:12,effici:13,still:[17,21,11,27,24,1,12,7],dynam:21,paramet:1,overwrit:[2,23],fix:[0,25,12,13,5,7],inadvert:[17,7],better:[27,22,4],window:12,html:[],restart:[27,13],persist:1,mail:[21,15,22,16],main:[10,7],might:[17,0,27,16,10,19,13,26,6,7],documents_januari:[17,7],them:[17,11,24,1,12,15,7],good:[0,25,13,5,14,7],synopsi:[8,17,0,11,23,10,25,3,4,26,6,20],thei:[17,0,18,21,25,12,5,14,7],python:[27,22,1],promin:13,safe:[17,14,7],fuse4bsd:22,dai:[5,25],initi:[],dat:[5,25],terminolog:[9,1],therebi:12,instead:[17,11,24,1,12,13,7],interrupt:15,potenti:12,now:[0,22,25,12,5,7],bigger:[],choic:22,term:[5,25,1],"__version__":22,somewher:[8,17,0,11,23,10,3,26,6,20,7],name:[17,23,1,25,2,12,5,14,20,27,7],joesdomain:[],authent:[8,9,20,19,23,1,2,12,3,15,27],achiev:[5,17,21,27,7],mode:[27,20],each:[5,25,27,20,1],debug:[8,17,0,20,11,23,10,24,19,25,2,3,4,26,6,15,27],found:[14,1],beneath:7,confid:[],side:1,mean:[21,12,13],compil:27,s3ql:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27],due:[12,13],mkfifo:27,chunk:13,hard:[14,21],idea:[5,25,0,7,13],procedur:[17,7],realli:[8,17,0,20,11,23,10,24,19,25,2,3,4,26,6,14,15,27,13],contrib:[5,14,27],meta:[26,14,7],significantli:[5,22,1],year:[5,25,21,12],s3_copi:[5,9],happen:[0,12,7,1],todo:[],special:[17,7],out:[21,12,13],lzma:[21,20,22,27],ftp:[],shown:12,network:[17,14,27,21,7],space:[17,21,7,13],open:[11,27,24,13],newli:13,log_fifo:27,content:[17,21,27,0,1,3,7],suitabl:[21,22],rational:0,internet:[21,27],print:[8,17,0,20,22,11,23,10,24,19,25,2,3,4,26,6,15,27],eth0:27,bla:27,correct:[5,25],common:[17,0,12,7],foreground:[27,20],shut:[5,11,24,13],after:[17,0,21,1,12,14,15,7],insid:17,advanc:[9,7],migrat:5,documents_janurai:[],manipul:21,situat:[14,13],given:13,free:[21,12],standard:[5,25,21,12,27],inod:13,reason:[10,27,24,1,25,12,13,5,14,19,7],base:27,theori:[12,13],usual:27,ask:[22,23,2,12,15,13],org:27,"byte":[],likelihood:12,afterward:14,bash:14,care:[17,21,7,13],her:[0,7],thread:[5,27,20],befor:[17,21,11,27,24,1,12,13,7],could:[17,7],success:[],refus:[5,25],keep:[0,27,1,25,5,7],recov:[0,7],thing:[15,3],length:12,rais:[27,20],place:[27,12],perman:[9,0,27,12,14,7],pycrypto:22,principl:27,confus:13,neglect:1,first:[21,22,27,1,12,14,15,13],origin:[],softwar:27,rang:[5,25],becaus:[17,21,27,1,12,5,7],directli:12,malici:[0,7],carri:21,onc:[0,11,27,21,24,12,13,14,20,7],clariti:[],s3q:[],number:[21,22,27,25,13,4,5,14,20,7],capac:21,restrict:[26,27,20],date:[5,25],instruct:[27,22],alreadi:[17,22,7,13],done:[27,12],wrapper:[5,4],llfuse:22,stabl:21,miss:[25,1],s3c:12,size:[21,23,2,13,26,14,20,27,7],differ:[17,21,22,27,1,26,3,5,14,15,7],convent:[8,17,0,11,23,10,3,26,6,20,13],script:[5,14,27,13],profil:[27,20],unknown:21,interact:3,s3qllock:[9,0,18,10,5,14,7],system:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,19,20,21,22,23,24,25,26,27],least:[22,27,25,12,5,20],parallel:[9,27,4,5,14,20],checkpoint:1,attack:[0,7,13],necessarili:12,demonstr:5,s3qlrm:[9,0,18,10,25,5,7],termin:[11,24,13,1],lost:[12,1],john:12,"final":7,store:[8,9,21,20,19,23,1,25,2,12,3,5,14,15,27,13],low:[21,12],servi:13,luckili:13,consol:12,option:[0,2,3,4,5,6,7,8,10,11,13,14,15,17,19,20,22,23,24,25,26,27],especi:[14,21],shelf:[0,7],tool:12,copi:[17,9,21,1,13,4,5,14,15,7],specifi:[8,20,19,23,27,1,25,2,12,3,5,15,26,13],arbitrari:[14,12],part:[21,27,1,25,4,5],pars:1,mostli:[],consult:[8,23,20,3],exactli:[5,17,25,7],than:[10,27,1,25,13,5,14,7],wide:[27,22],target:[8,17,20,27,19,3,14,15,7],cachedir:[8,15,23,19,2,3,20,27],whenev:[14,22,27,1],prevent:[2,0,23,7],remov:[17,9,0,22,27,10,25,26,7,5,6,13],eleg:21,tree:[17,9,0,10,4,5,21,7],second:[27,1,25,12,5,14,20],structur:[21,15],exampl:[21,1,25,13,5,14,7],matter:[26,14,15],temporarili:27,friend:14,video:27,minut:[5,25],led:1,pre:27,unencrypt:[2,23],sai:[17,7],comput:[5,25,21,27],entail:[],januari:[17,7],plaintext:[],explicit:14,ram:27,mind:1,argument:[26,15,3],peculiar:12,"13th":21,packag:[5,22],expir:27,increment:[],disadvantag:[17,7],need:[17,20,22,27,1,25,12,5,15,7],seem:14,exclud:[0,7],paramiko:[],caus:[12,1],equival:[5,25],irrepar:15,destroi:[0,7],moreov:7,blindli:13,atim:13,accuraci:[5,25],note:[17,9,21,22,11,27,24,25,12,7,4,5,14,15,13],also:[8,17,0,18,11,23,10,24,1,25,26,12,21,3,4,5,6,14,20,27,7],builtin:22,denomin:12,take:[17,21,13,3,26,14,15,7],which:[17,10,22,21,1,25,12,13,5,7],discrep:[],transmit:[],environ:22,uplink:5,singl:[14,20,27],swordfish:1,compat:[9,12,13],begin:1,sure:[20,22,27,12,14,15,13],unless:[2,0,23,7],distribut:[5,27,22],plenti:[],normal:[27,20,1],buffer:13,previou:[5,15,12],compress:[9,21,27,5,20,7],most:[8,17,20,19,27,1,25,12,7,3,5,15,13],beta:21,said:[],rigor:[5,25],plan:[],choos:[],homepag:[8,17,0,11,23,10,3,26,6,20],"class":[8,17,0,11,23,10,25,26,3,4,5,6,20],independ:[21,20,27],simplic:21,renam:21,correctli:[13,19],ship:[25,27,22,4],url:[8,20,19,23,1,2,12,3,15,27],doc:[8,17,0,11,23,10,26,3,5,6,20],clear:[17,15,7],later:[17,0,22,1,12,13,7],cover:14,drive:[0,7],destruct:[0,7],doe:[21,27,24,1,12,13],declar:21,snapshot:[17,9,21,26,7],runtim:[9,7],determin:[5,27,1],sourceforg:[],occasion:12,region:[5,12],hmac:21,gracefulli:[5,25],myawssecretaccesskei:[],theoret:14,show:25,carefulli:1,random:[],syntax:[27,19,2,26,15,7],connect:[14,12,27,21],permiss:[5,21,20,27],bucketnam:12,newest:[14,3],anywai:[],rotat:[8,15,27,19,3,20],redirect:27,current:[5,15,12,25,13],onli:[17,25,0,20,22,11,27,10,24,21,26,12,7,4,5,6,14,15,13],slow:13,locat:[8,17,0,11,23,10,1,3,26,6,20,7],execut:[5,15,22,7],transact:21,configur:[9,27,7],activ:[8,17,0,20,19,11,23,10,24,1,25,2,12,3,4,26,6,15,27,7],state:[5,25,21,13],haven:[],authinfo:[],latenc:[14,21,4],suppos:12,rich:12,factor:[5,12],folder:[17,22,7],local:[8,9,0,22,17,11,23,10,12,21,3,26,6,14,20,27],defin:[5,25,27,1],contribut:[5,9],variou:[26,15,3],get:[8,9,0,20,22,19,23,27,1,25,2,16,3,5,15,26,7],googlegroup:16,nasti:[0,7],stop:27,autom:[5,21],regularli:[17,7],ssl:13,s3rr:[],cannot:[5,25],ssh:[9,14,12],report:[21,22,16,1],reconstruct:[5,25,1],requir:[8,21,1,27,19,25,26,5,22,20,13],myawsaccesskeyid:[],reveal:21,enabl:[],dramat:14,intrins:14,method:[27,22],provid:[21,22,27,1,12,13],bad:14,statist:[6,7,9],though:[27,12,1],contain:[22,27,25,3,5,15],userspac:[],nowher:[],where:[26,15,13,3,1],caches:[26,27,20,7],wiki:[22,16],kernel:22,set:[5,14,20,27],bucket1:1,bucket3:1,bucket2:1,startup:27,maximum:[5,2,23,20,27],see:[8,17,0,20,11,23,10,1,25,26,12,7,3,4,5,6,14,15,27,13],num:[27,20],s3qlcp:[5,9,17,18,7],fail:[12,22,13],close:[11,24],optimum:14,whatsoev:[0,21,7],best:[14,12,22,27],concern:1,infinit:21,awar:1,statu:[8,9,0,17,11,23,10,25,12,21,3,4,26,6,20,13],detect:[21,12,1],extend:21,inconveni:21,hopefulli:13,databas:21,boundari:[5,25],label:[2,23],favor:21,enough:[14,20,27,13],between:[5,25,27,20],"import":[5,22,1],neither:[0,7],across:21,attribut:21,amazon:[9,21,1,12,5,13],august:[],kei:[21,12],weak:[],southeast:[],job:[5,27],hardlink:[17,21,7],joe:[5,1],expire_backup:[5,9,18,25],solv:14,come:[5,14,21],local0:27,addit:[17,10,27,21,1,3,26,15,7],both:[17,21,22,7,1],protect:[0,21,7],accident:[2,0,23,7,13],last:1,irregular:[5,25],extens:21,someth:[17,7],howev:[17,0,27,21,25,12,13,5,14,7],alon:1,lazi:[11,24],against:[0,21,7],configpars:1,etc:[17,27,7,13],instanc:14,freeli:27,corrupt:1,com:[8,17,0,22,11,23,10,25,3,4,26,6,14,20,16],pcp:[5,9,14,18,4],load:27,simpli:[0,7],figur:1,inspir:[17,7],buti:[],period:[0,7],insist:14,batch:[8,27,19],written:[21,20,27,1],littl:[27,12,22],shutdown:27,linux:[27,22],averag:12,guid:[8,9,18,23,3,20],assum:27,damag:[5,25,15,1],quit:22,worthless:[0,7],strong:[8,17,0,11,23,10,25,3,4,26,6,20],nikolau:27,west:[],devic:27,three:[27,12,22],been:[0,19,11,27,21,24,1,25,12,7,5,14,15,13],mark:[8,19],compon:21,secret:12,much:4,interpret:27,interest:7,subscrib:16,monthli:[17,7],immedi:[10,11,27,24,12,7],strategi:[5,25],legaci:12,infect:[0,7],upstart:[5,27,20],great:[0,7],ani:[8,17,0,22,23,21,1,25,2,12,16,5,7],rsync_arg:14,zero:14,understand:[27,20],togeth:1,els:15,tradition:[0,7],s3qlstat:[6,18,7,13,9],present:12,"case":[17,0,22,21,1,25,5,14,15,7],replic:[5,17,21,7],trojan:[0,7],ident:[17,21,7,1],look:14,gnu:13,solid:21,plain:[2,23],servic:[21,12,1],zlib:[27,20],histor:1,trick:[9,14],documents_februari:[17,7],invok:[26,15],abov:[5,25,1],error:[8,9,0,20,17,11,23,10,1,25,21,3,4,26,6,19,13],login:[12,1],invoc:[5,25],ahax:4,loos:[12,1],jibbadup:[],earli:13,runlevel:27,argpars:22,have:[17,0,20,22,11,23,16,21,24,1,25,2,12,7,4,5,14,15,27,13],advantag:[5,25,0,7],stdout:27,almost:14,mtime:13,therefor:[5,27,12,13,1],remount:12,worri:13,destin:[17,14,7,4],exit:[8,17,0,20,11,23,10,24,19,25,2,3,4,26,6,15,27],gsutil:[],conf:[5,9],incom:[],revis:[15,3],dedic:[27,13],sever:[21,22,27,1,12,4,5,14,20],tamper:[5,25],unmount:[9,11,27,24,1,12,5,14,19,13],develop:[9,21,12],minim:21,perform:[9,21,22,27,1,26,12,3,4,5,14,15,13],media:12,make:[17,0,20,22,27,21,1,25,12,13,5,14,15,7],flushcach:[26,7],same:[17,27,1,25,12,5,14,15,7],"while":[0,22,27,21,24,12,26,7],handl:[17,21,27,25,12,13,5,14,7],inconsist:13,unexpectedli:13,split:21,auto:[27,20],pai:12,document:[8,17,0,11,23,10,3,26,6,20,7],infer:[5,25],complet:[17,22,27,1,25,26,12,5,20,7],week:[5,25],geograph:12,archiv:[21,12],hostnam:12,closest:12,lie:17,optim:27,keyr:[],confidenti:[],upon:13,effect:[17,21,7,1],cycl:[5,25],solut:5,remot:[14,12,21],fuse:[27,13],temporari:12,user:[8,9,0,18,22,17,11,23,10,24,19,12,21,3,26,6,14,20,27,7],mani:[5,25,21,1],extern:[0,12,7],encrypt:[21,27,1,2,3,23],typic:[17,7],recent:[5,25,27,15,13],gss:12,appropri:27,kept:[8,15,27,19,3,20],older:15,nevertheless:[5,1],entri:[27,20,13,1],thu:1,irrelev:13,well:[0,20,27,10,1,25,5,15,7],without:[8,17,0,21,19,12,13,14,15,7],command:[0,1,2,3,4,5,6,7,8,9,10,11,13,15,17,18,19,20,22,23,24,25,26,27],thi:[0,1,2,3,4,5,6,7,8,10,12,13,14,15,16,17,19,20,21,22,23,24,25,26,27],filesystem:[27,19,2,14,20,23],gzip:21,credenti:[8,20,1,23,19,2,3,15,27],spend:14,left:13,compromis:14,identifi:[12,1],just:[8,17,0,20,11,23,10,24,19,25,2,12,3,4,26,6,14,15,27,7],less:[13,1],conform:21,tip:[9,14],lowest:12,obtain:[],rest:14,bandwidth:[5,21],touch:[17,7],openstack:21,passphras:[9,15,3,1],roughli:27,speed:[5,27],yet:[12,13],web:12,viru:[0,7],detach:[11,24],homedir:[],easi:14,hint:[26,15,3],trigger:[12,7],point:[21,11,27,1,12,20],had:[5,17,25,7],except:1,param:15,thousand:[27,12,13],add:[0,7],valid:[12,7],nor:[0,7],versa:[17,7],input:[8,19],logger:27,subsequ:[5,25,12],match:12,bin:[14,22],applic:[13,1],transpar:21,preserv:[5,21],big:[14,13],regard:[17,7],exception:[],traffic:12,know:[5,17,25,7,13],background:[11,24],amp:14,bit:21,password:[2,23,12,1],recurs:[5,9,10,7],presum:1,like:[17,0,27,21,12,13,4,20,7],loss:[12,1],daemon:[27,20],ctime:13,specif:[0,22,1,12,13,26,15,7],header:1,should:[8,17,0,20,22,19,11,23,10,24,1,7,3,26,6,14,15,27,13],anyth:[],manual:[11,22,24,16],resolv:[12,16],noth:[0,7],princip:13,necessari:15,either:[12,13,3,26,15,7],output:[8,17,0,20,11,23,10,24,19,25,2,3,4,26,6,15,27],per:27,page:18,yyyi:[5,25],imagin:[5,25,0,7],right:14,old:[8,0,20,22,27,19,25,12,3,5,15,7],often:[14,1],deal:[],ll23bi:1,interv:[5,25,14,20,27],creation:[2,23,9],some:[8,17,0,20,22,11,23,10,24,1,25,26,12,21,3,4,5,6,19,13],umount:[9,18,11,27,24,13],self:22,certain:[],strongest:[],"export":[27,20,13],flush:[27,7],guarante:[5,25,12],server:[14,12,13],librari:[22,13],"24h":[27,20],rsync:[17,0,13,4,5,14,7],backend:[8,9,17,11,23,24,1,12,7,3,26,14,20,27,13],confirm:[10,7],stronger:12,freebsd:22,avoid:[5,25],exec:27,definit:[5,25],februari:[17,7],protocol:1,usernam:1,equal:27,leav:13,slash:12,cif:27,duplic:[17,21,1,12,5,7],creep:21,refer:[18,1],machin:[],core:[5,27],plu:1,object:[20,12,27],run:[0,20,22,19,27,1,25,26,13,3,5,15,7],itself:[12,22,13,1],power:5,certif:[12,13],reach:[8,15,27,19,3,20],intellig:[5,25,21],view:18,usag:[5,17,27,25,7],symlink:21,speak:27,host:[],unreason:13,although:[21,10,7,13],eventu:12,bi23ll:1,immut:[9,0,10,5,21,14,7],impos:[],stage:[],sshf:[14,12],comparison:[],deflat:21,actual:[15,14,12,22,13],proce:22,memori:27,http:[8,17,0,22,11,23,10,25,12,3,4,26,6,20],storebackup:[17,7],acl:21,messag:[8,15,23,19,2,3,20,27],fals:[27,20],commit:[24,13],backup:[17,9,0,27,21,1,25,12,3,5,14,15,7],disabl:[8,15,27,19,3,20],block:[21,11,23,24,1,2,13,26,27,7],repair:8,client:13,real:12,encount:[16,13],xyz:[],within:[7,1],encod:27,automat:[5,9,27,12,15],two:[17,21,15,22,7],down:[5,11,24,13],ahv:14,authinfo2:[8,15,23,19,2,3,20,27],ensur:[5,25,14,21],chang:[17,9,0,27,21,12,7,3,26,15,13],insuffici:13,storag:[8,9,0,20,19,17,11,23,21,24,1,2,12,7,3,5,14,15,27,13],your:[8,17,0,18,22,11,23,10,25,26,12,3,5,6,14,20,27,7],durabl:1,manag:[9,12,15],east:[],fast:[9,21,27,7],fusermount:[11,24,13],prepar:26,wai:[0,27,21,1,12,13,14,7],transfer:[5,14,21],support:[17,21,27,12,13,20,7],question:16,s3_backup:[5,9],"long":[5,25,14,13,1],avail:[8,0,20,23,25,12,7,3,5,15,27,13],start:[22,27,1,12,4,5,14],reli:[13,1],quiet:[8,17,0,20,11,23,10,24,19,25,2,3,4,26,6,15,27],includ:[22,21,12,18,13],lot:14,"var":12,succeed:[8,17,0,11,23,10,25,3,4,26,6,20],individu:[27,20,4],"function":15,properli:5,tracker:[21,22,16],form:[5,25,12,1],offer:[17,12,7],forc:[8,2,23,19],basic:[],continu:24,sigstop:[27,20],satur:27,measur:5,newer:[12,22],don:[27,20,13],line:[26,21],bug:[21,22,16,13],faster:[10,12,7],info:[8,15,27,19,3,26,20],commun:1,made:[17,21,0,14,7],furthermor:[21,1],consist:[12,1],possibl:[0,22,27,21,12,13,14,15,7],"default":[8,20,23,19,25,2,3,4,5,15,27],bucket:[9,23,1,2,12,3,5,15,27,13],displai:13,tell:[5,25,27],asynchron:21,authfil:[8,20,1,23,19,2,3,15,27],below:[12,7],limit:[17,0,27,10,1,26,12,7,5,6,20,13],unnot:12,problem:[8,17,0,16,21,1,25,12,7,5,14,13],similar:27,expect:[27,12,1],featur:[17,9,0,21,1,12,7],creat:[17,0,20,23,10,25,2,12,7,5,21,15,27,13],classic:[],retriev:[14,12,27,20,4],dure:[15,20,12,27],day_hour:[5,25],decrypt:1,s3qlctrl:[26,9,14,18,7],strongli:[5,25],workaround:14,decreas:4,file:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,19,20,21,23,24,25,26,27],home:[5,14,12],request:12,exist:[17,0,23,10,1,25,2,12,5,7],improv:[5,9,14,4],mybucket:14,check:[8,9,22,27,19,12,15,13],probabl:[21,25,5,14,15,7],otherwis:[27,20],again:[0,12,7,13,24],readi:12,relatim:13,umask:27,googl:[8,9,0,22,17,11,23,10,25,12,21,3,4,26,6,20],when:[8,0,20,19,27,21,1,25,12,7,3,4,5,14,15,13],detail:[5,14,12,13],prepend:12,field:[],other:[17,0,22,27,10,21,12,16,26,6,20,7],futur:[17,0,10,13,26,6,7],rememb:[],test:[21,22],you:[17,0,20,22,19,11,23,16,10,24,1,25,2,12,7,5,14,15,27,13],shrink:21,rid:[0,7],particular:1,variabl:22,intend:[],clean:[8,19],fulli:13,mountpoint:[27,24,12,26,6,14,20,7],"return":[8,17,0,11,23,10,24,25,3,4,26,6,20,13],fsck:[8,9,18,19,27,1,12,15],briefli:[],releas:21,track:[5,25],log:[8,20,19,27,1,12,3,26,15,7],consid:[12,16,1],sql:24,noleaf:13,dd_hh:[5,25],pool:[],stai:[27,20],reduc:[5,12],infrequ:1,longer:[21,0,24,25,13,5,15,7],algorithm:[5,9,21,20,27],vice:[17,7],directori:[0,1,2,3,4,5,7,8,10,11,12,13,14,15,17,19,20,21,22,23,25,27],reliabl:[9,12,1],descript:[8,17,0,11,23,10,25,3,4,26,6,14,20,27],save:[25,14,21,1],rule:14,sftp:[9,12],depth:[],ignor:[],back:[0,7],time:[8,0,20,19,23,21,1,25,2,12,7,3,5,14,15,27,13],backward:[],s3qladm:[9,15,18,3],daili:[]},objtypes:{},titles:["The <strong class=\"program\">s3qllock</strong> command","General Information","File System Creation","The <strong class=\"program\">s3qladm</strong> command","The <strong class=\"program\">pcp</strong> command","Contributed Programs","The <strong class=\"program\">s3qlstat</strong> command","Advanced S3QL Features","The <strong class=\"program\">fsck.s3ql</strong> command","S3QL User&#8217;s Guide","The <strong class=\"program\">s3qlrm</strong> command","The <strong class=\"program\">umount.s3ql</strong> command","Storage Backends","Known Issues","Tips &amp; Tricks","Managing Buckets","Further Resources / Getting Help","The <strong class=\"program\">s3qlcp</strong> command","Manpages","Checking for Errors","The <strong class=\"program\">mount.s3ql</strong> command","About S3QL","Installation","The <strong class=\"program\">mkfs.s3ql</strong> command","Unmounting","The <strong class=\"program\">expire_backups</strong> command","The <strong class=\"program\">s3qlctrl</strong> command","Mounting"],objnames:{},filenames:["man/lock","general","mkfs","man/adm","man/pcp","contrib","man/stat","special","man/fsck","index","man/rm","man/umount","backends","issues","tips","adm","resources","man/cp","man/index","fsck","man/mount","about","installation","man/mkfs","umount","man/expire_backups","man/ctrl","mount"]}) \ No newline at end of file
diff --git a/doc/html/special.html b/doc/html/special.html
index 768f100..1d795cd 100644
--- a/doc/html/special.html
+++ b/doc/html/special.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Advanced S3QL Features &mdash; S3QL 1.0.1 documentation</title>
+ <title>Advanced S3QL Features &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Unmounting" href="umount.html" />
<link rel="prev" title="Mounting" href="mount.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="mount.html" title="Mounting"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -96,44 +97,40 @@
<h1>Advanced S3QL Features<a class="headerlink" href="#advanced-s3ql-features" title="Permalink to this headline">¶</a></h1>
<div class="section" id="snapshotting-and-copy-on-write">
<span id="s3qlcp"></span><h2>Snapshotting and Copy-on-Write<a class="headerlink" href="#snapshotting-and-copy-on-write" title="Permalink to this headline">¶</a></h2>
-<p>The command <tt class=" docutils literal"><span class="pre">s3qlcp</span></tt> can be used to duplicate a directory tree without
-physically copying the file contents. This is possible due to the data
-de-duplication feature of S3QL.</p>
-<p>The syntax of <tt class=" docutils literal"><span class="pre">s3qlcp</span></tt> is:</p>
+<p>The command <tt class="docutils literal"><span class="pre">s3qlcp</span></tt> can be used to duplicate a directory tree without
+physically copying the file contents. This is made possible by the
+data de-duplication feature of S3QL.</p>
+<p>The syntax of <tt class="docutils literal"><span class="pre">s3qlcp</span></tt> is:</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3qlcp </span><span class="ge">[options]</span><span class="l"> </span><span class="nv">&lt;src&gt;</span><span class="l"> </span><span class="nv">&lt;target&gt;</span><span class="l"></span>
</pre></div>
</div>
-<p>This will replicate the contents of the directory <tt class=" docutils literal"><span class="pre">&lt;src&gt;</span></tt> in the
-directory <tt class=" docutils literal"><span class="pre">&lt;target&gt;</span></tt>. <tt class=" docutils literal"><span class="pre">&lt;src&gt;</span></tt> has to be an existing directory and
-<tt class=" docutils literal"><span class="pre">&lt;target&gt;</span></tt> must not exist. Moreover, both directories have to be
+<p>This will replicate the contents of the directory <tt class="docutils literal"><span class="pre">&lt;src&gt;</span></tt> in the
+directory <tt class="docutils literal"><span class="pre">&lt;target&gt;</span></tt>. <tt class="docutils literal"><span class="pre">&lt;src&gt;</span></tt> has to be an existing directory and
+<tt class="docutils literal"><span class="pre">&lt;target&gt;</span></tt> must not exist. Moreover, both directories have to be
within the same S3QL file system.</p>
<p>The replication will not take any additional space. Only if one of
directories is modified later on, the modified data will take
additional storage space.</p>
-<p><tt class=" docutils literal"><span class="pre">s3qlcp</span></tt> can only be called by the user that mounted the file system
-and (if the file system was mounted with <tt class=" docutils literal"><span class="pre">--allow-other</span></tt> or <tt class=" docutils literal"><span class="pre">--allow-root</span></tt>)
+<p><tt class="docutils literal"><span class="pre">s3qlcp</span></tt> can only be called by the user that mounted the file system
+and (if the file system was mounted with <tt class="docutils literal"><span class="pre">--allow-other</span></tt> or <tt class="docutils literal"><span class="pre">--allow-root</span></tt>)
the root user. This limitation might be removed in the future (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=155">issue 155</a>).</p>
<p>Note that:</p>
<ul class="simple">
<li>After the replication, both source and target directory will still
-be completely ordinary directories. You can regard <tt class=" docutils literal"><span class="pre">&lt;src&gt;</span></tt> as a
-snapshot of <tt class=" docutils literal"><span class="pre">&lt;target&gt;</span></tt> or vice versa. However, the most common
-usage of <tt class=" docutils literal"><span class="pre">s3qlcp</span></tt> is to regularly duplicate the same source
-directory, say <tt class=" docutils literal"><span class="pre">documents</span></tt>, to different target directories. For a
+be completely ordinary directories. You can regard <tt class="docutils literal"><span class="pre">&lt;src&gt;</span></tt> as a
+snapshot of <tt class="docutils literal"><span class="pre">&lt;target&gt;</span></tt> or vice versa. However, the most common
+usage of <tt class="docutils literal"><span class="pre">s3qlcp</span></tt> is to regularly duplicate the same source
+directory, say <tt class="docutils literal"><span class="pre">documents</span></tt>, to different target directories. For a
e.g. monthly replication, the target directories would typically be
-named something like <tt class=" docutils literal"><span class="pre">documents_Januray</span></tt> for the replication in
-January, <tt class=" docutils literal"><span class="pre">documents_February</span></tt> for the replication in February etc.
+named something like <tt class="docutils literal"><span class="pre">documents_January</span></tt> for the replication in
+January, <tt class="docutils literal"><span class="pre">documents_February</span></tt> for the replication in February etc.
In this case it is clear that the target directories should be
regarded as snapshots of the source directory.</li>
<li>Exactly the same effect could be achieved by an ordinary copy
-program like <tt class=" docutils literal"><span class="pre">cp</span> <span class="pre">-a</span></tt>. However, this procedure would be orders of
-magnitude slower, because <tt class=" docutils literal"><span class="pre">cp</span></tt> would have to read every file
+program like <tt class="docutils literal"><span class="pre">cp</span> <span class="pre">-a</span></tt>. However, this procedure would be orders of
+magnitude slower, because <tt class="docutils literal"><span class="pre">cp</span></tt> would have to read every file
completely (so that S3QL had to fetch all the data over the network
from the backend) before writing them into the destination folder.</li>
-<li>Before starting with the replication, S3QL has to flush the local
-cache. So if you just copied lots of new data into the file system
-that has not yet been uploaded, replication will take longer than
-usual.</li>
</ul>
<div class="section" id="snapshotting-vs-hardlinking">
<h3>Snapshotting vs Hardlinking<a class="headerlink" href="#snapshotting-vs-hardlinking" title="Permalink to this headline">¶</a></h3>
@@ -159,17 +156,17 @@ any backup program.</p>
<div class="section" id="getting-statistics">
<span id="s3qlstat"></span><h2>Getting Statistics<a class="headerlink" href="#getting-statistics" title="Permalink to this headline">¶</a></h2>
<p>You can get more information about a mounted S3QL file system with the
-<tt class=" docutils literal"><span class="pre">s3qlstat</span></tt> command. It has the following syntax:</p>
+<tt class="docutils literal"><span class="pre">s3qlstat</span></tt> command. It has the following syntax:</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3qlstat </span><span class="ge">[options]</span><span class="l"> </span><span class="nv">&lt;mountpoint&gt;</span><span class="l"></span>
</pre></div>
</div>
<p>Probably the most interesting numbers are the total size of your data,
the total size after duplication, and the final size after
de-duplication and compression.</p>
-<p><tt class=" docutils literal"><span class="pre">s3qlstat</span></tt> can only be called by the user that mounted the file system
-and (if the file system was mounted with <tt class=" docutils literal"><span class="pre">--allow-other</span></tt> or <tt class=" docutils literal"><span class="pre">--allow-root</span></tt>)
+<p><tt class="docutils literal"><span class="pre">s3qlstat</span></tt> can only be called by the user that mounted the file system
+and (if the file system was mounted with <tt class="docutils literal"><span class="pre">--allow-other</span></tt> or <tt class="docutils literal"><span class="pre">--allow-root</span></tt>)
the root user. This limitation might be removed in the future (see <a class="reference external" href="http://code.google.com/p/s3ql/issues/detail?id=155">issue 155</a>).</p>
-<p>For a full list of available options, run <tt class=" docutils literal"><span class="pre">s3qlstat</span> <span class="pre">--help</span></tt>.</p>
+<p>For a full list of available options, run <tt class="docutils literal"><span class="pre">s3qlstat</span> <span class="pre">--help</span></tt>.</p>
</div>
<div class="section" id="immutable-trees">
<span id="s3qllock"></span><h2>Immutable Trees<a class="headerlink" href="#immutable-trees" title="Permalink to this headline">¶</a></h2>
@@ -226,13 +223,13 @@ be removed entirely and immediately.</p>
</div>
<div class="section" id="runtime-configuration">
<span id="s3qlctrl"></span><h2>Runtime Configuration<a class="headerlink" href="#runtime-configuration" title="Permalink to this headline">¶</a></h2>
-<p>The <tt class=" docutils literal"><span class="pre">s3qlctrl</span></tt> can be used to control a mounted S3QL file system. Its
+<p>The <tt class="docutils literal"><span class="pre">s3qlctrl</span></tt> can be used to control a mounted S3QL file system. Its
syntax is</p>
<div class="highlight-commandline"><div class="highlight"><pre><span class="l">s3qlctrl </span><span class="ge">[options]</span><span class="l"> </span><span class="nv">&lt;action&gt;</span><span class="l"> </span><span class="nv">&lt;mountpoint&gt;</span><span class="l"> ...</span>
</pre></div>
</div>
-<p><tt class=" docutils literal"><span class="pre">&lt;mountpoint&gt;</span></tt> must be the location of a mounted S3QL file system.
-For a list of valid options, run <tt class=" docutils literal"><span class="pre">s3qlctrl</span> <span class="pre">--help</span></tt>. <tt class=" docutils literal"><span class="pre">&lt;action&gt;</span></tt>
+<p><tt class="docutils literal"><span class="pre">&lt;mountpoint&gt;</span></tt> must be the location of a mounted S3QL file system.
+For a list of valid options, run <tt class="docutils literal"><span class="pre">s3qlctrl</span> <span class="pre">--help</span></tt>. <tt class="docutils literal"><span class="pre">&lt;action&gt;</span></tt>
may be either of:</p>
<blockquote>
<div><table class="docutils field-list" frame="void" rules="none">
@@ -269,7 +266,7 @@ been flushed.</td>
<li class="right" >
<a href="mount.html" title="Mounting"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/tips.html b/doc/html/tips.html
index 6560a99..6f0d684 100644
--- a/doc/html/tips.html
+++ b/doc/html/tips.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Tips &amp; Tricks &mdash; S3QL 1.0.1 documentation</title>
+ <title>Tips &amp; Tricks &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Known Issues" href="issues.html" />
<link rel="prev" title="Contributed Programs" href="contrib.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="contrib.html" title="Contributed Programs"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -58,6 +59,7 @@
<li class="toctree-l1"><a class="reference internal" href="fsck.html">Checking for Errors</a></li>
<li class="toctree-l1"><a class="reference internal" href="contrib.html">Contributed Programs</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="">Tips &amp; Tricks</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#ssh-backend">SSH Backend</a></li>
<li class="toctree-l2"><a class="reference internal" href="#permanently-mounted-backup-file-system">Permanently mounted backup file system</a></li>
<li class="toctree-l2"><a class="reference internal" href="#improving-copy-performance">Improving copy performance</a></li>
</ul>
@@ -91,8 +93,21 @@
<div class="section" id="tips-tricks">
<h1>Tips &amp; Tricks<a class="headerlink" href="#tips-tricks" title="Permalink to this headline">¶</a></h1>
+<div class="section" id="ssh-backend">
+<span id="ssh-tipp"></span><h2>SSH Backend<a class="headerlink" href="#ssh-backend" title="Permalink to this headline">¶</a></h2>
+<p>By combining S3QL&#8217;s local backend with <a class="reference external" href="http://fuse.sourceforge.net/sshfs.html">sshfs</a>, it is possible to store an
+S3QL file system on arbitrary SSH servers: first mount the remote
+target directory into the local filesystem,</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">sshfs user@my.server.com:/mnt/s3ql /mnt/sshfs</span>
+</pre></div>
+</div>
+<p>and then give the mountpoint to S3QL as a local destination:</p>
+<div class="highlight-commandline"><div class="highlight"><pre><span class="l">mount.s3ql local:///mnt/sshfs/mybucket /mnt/s3ql</span>
+</pre></div>
+</div>
+</div>
<div class="section" id="permanently-mounted-backup-file-system">
-<span id="copy-performance"></span><h2>Permanently mounted backup file system<a class="headerlink" href="#permanently-mounted-backup-file-system" title="Permalink to this headline">¶</a></h2>
+<h2>Permanently mounted backup file system<a class="headerlink" href="#permanently-mounted-backup-file-system" title="Permalink to this headline">¶</a></h2>
<p>If you use S3QL as a backup file system, it can be useful to mount the
file system permanently (rather than just mounting it for a backup and
unmounting it afterwards). Especially if your file system becomes
@@ -110,7 +125,12 @@ to zero).</li>
</ul>
</div>
<div class="section" id="improving-copy-performance">
-<h2>Improving copy performance<a class="headerlink" href="#improving-copy-performance" title="Permalink to this headline">¶</a></h2>
+<span id="copy-performance"></span><h2>Improving copy performance<a class="headerlink" href="#improving-copy-performance" title="Permalink to this headline">¶</a></h2>
+<div class="admonition note">
+<p class="first admonition-title">Note</p>
+<p class="last">The following applies only when copying data <strong>from</strong> an S3QL file
+system, <strong>not</strong> when copying data <strong>to</strong> an S3QL file system.</p>
+</div>
<p>If you want to copy a lot of smaller files <em>from</em> an S3QL file system
(e.g. for a system restore) you will probably notice that the
performance is rather bad.</p>
@@ -170,7 +190,7 @@ details.</p>
<li class="right" >
<a href="contrib.html" title="Contributed Programs"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/html/umount.html b/doc/html/umount.html
index 23e4c45..32bd358 100644
--- a/doc/html/umount.html
+++ b/doc/html/umount.html
@@ -8,7 +8,7 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Unmounting &mdash; S3QL 1.0.1 documentation</title>
+ <title>Unmounting &mdash; S3QL 1.2 documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
@@ -16,7 +16,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
- VERSION: '1.0.1',
+ VERSION: '1.2',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
@@ -26,7 +26,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
- <link rel="top" title="S3QL 1.0.1 documentation" href="index.html" />
+ <link rel="top" title="S3QL 1.2 documentation" href="index.html" />
<link rel="next" title="Checking for Errors" href="fsck.html" />
<link rel="prev" title="Advanced S3QL Features" href="special.html" />
</head>
@@ -40,7 +40,7 @@
<li class="right" >
<a href="special.html" title="Advanced S3QL Features"
accesskey="P">previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="sphinxsidebar">
@@ -49,6 +49,7 @@
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="about.html">About S3QL</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="general.html">General Information</a></li>
<li class="toctree-l1"><a class="reference internal" href="backends.html">Storage Backends</a></li>
<li class="toctree-l1"><a class="reference internal" href="mkfs.html">File System Creation</a></li>
<li class="toctree-l1"><a class="reference internal" href="adm.html">Managing Buckets</a></li>
@@ -98,8 +99,8 @@ is able to unmount it again. If you are root and want to unmount an
S3QL file system mounted by an ordinary user, you have to use the
<strong class="command">fusermount -u</strong> or <strong class="command">umount</strong> command instead. Note
that these commands do not block until all data has been uploaded, so
-if you use them instead of <tt class=" docutils literal"><span class="pre">umount.s3ql</span></tt> then you should manually wait
-for the <tt class=" docutils literal"><span class="pre">mount.s3ql</span></tt> process to terminate before shutting down the
+if you use them instead of <tt class="docutils literal"><span class="pre">umount.s3ql</span></tt> then you should manually wait
+for the <tt class="docutils literal"><span class="pre">mount.s3ql</span></tt> process to terminate before shutting down the
system.</p>
<p>The <strong class="command">umount.s3ql</strong> command accepts the following options:</p>
<blockquote>
@@ -124,8 +125,8 @@ background once all open files have been closed.</td></tr>
</tbody>
</table>
</div></blockquote>
-<p>If, for some reason, the <tt class=" docutils literal"><span class="pre">umount.sql</span></tt> command does not work, the file
-system can also be unmounted with <tt class=" docutils literal"><span class="pre">fusermount</span> <span class="pre">-u</span> <span class="pre">-z</span></tt>. Note that this
+<p>If, for some reason, the <tt class="docutils literal"><span class="pre">umount.sql</span></tt> command does not work, the file
+system can also be unmounted with <tt class="docutils literal"><span class="pre">fusermount</span> <span class="pre">-u</span> <span class="pre">-z</span></tt>. Note that this
command will return immediately and the file system may continue to
upload data in the background for a while longer.</p>
</div>
@@ -145,7 +146,7 @@ upload data in the background for a while longer.</p>
<li class="right" >
<a href="special.html" title="Advanced S3QL Features"
>previous</a> |</li>
- <li><a href="index.html">S3QL 1.0.1 documentation</a> &raquo;</li>
+ <li><a href="index.html">S3QL 1.2 documentation</a> &raquo;</li>
</ul>
</div>
<div class="footer">
diff --git a/doc/latex/manual.aux b/doc/latex/manual.aux
index 4f41234..5c7507a 100644
--- a/doc/latex/manual.aux
+++ b/doc/latex/manual.aux
@@ -36,331 +36,312 @@
\@writefile{toc}{\contentsline {section}{\numberline {2.2}Installing S3QL}{4}{section.2.2}}
\newlabel{installation:inst-s3ql}{{2.2}{4}{Installing S3QL\relax }{section.2.2}{}}
\newlabel{installation:installing-s3ql}{{2.2}{4}{Installing S3QL\relax }{section.2.2}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {3}Storage Backends}{5}{chapter.3}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {3}General Information}{5}{chapter.3}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{backends::doc}{{3}{5}{Storage Backends\relax }{chapter.3}{}}
-\newlabel{backends:storage-backends}{{3}{5}{Storage Backends\relax }{chapter.3}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {3.1}On Backend Reliability}{5}{section.3.1}}
-\newlabel{backends:on-backend-reliability}{{3.1}{5}{On Backend Reliability\relax }{section.3.1}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {3.2}The \texttt {authinfo} file}{6}{section.3.2}}
-\newlabel{backends:the-authinfo-file}{{3.2}{6}{The \texttt {authinfo} file\relax }{section.3.2}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {3.3}Consistency Guarantees}{6}{section.3.3}}
-\newlabel{backends:consistency-guarantees}{{3.3}{6}{Consistency Guarantees\relax }{section.3.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {3.3.1}Dealing with Eventual Consistency}{6}{subsection.3.3.1}}
-\newlabel{backends:dealing-with-eventual-consistency}{{3.3.1}{6}{Dealing with Eventual Consistency\relax }{subsection.3.3.1}{}}
-\newlabel{backends:eventual-consistency}{{3.3.1}{6}{Dealing with Eventual Consistency\relax }{subsection.3.3.1}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {3.4}The Amazon S3 Backend}{7}{section.3.4}}
-\newlabel{backends:the-amazon-s3-backend}{{3.4}{7}{The Amazon S3 Backend\relax }{section.3.4}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {3.5}The Local Backend}{8}{section.3.5}}
-\newlabel{backends:the-local-backend}{{3.5}{8}{The Local Backend\relax }{section.3.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {3.6}The SFTP Backend}{8}{section.3.6}}
-\newlabel{backends:the-sftp-backend}{{3.6}{8}{The SFTP Backend\relax }{section.3.6}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {4}File System Creation}{9}{chapter.4}}
+\newlabel{general:general-information}{{3}{5}{General Information\relax }{chapter.3}{}}
+\newlabel{general::doc}{{3}{5}{General Information\relax }{chapter.3}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {3.1}Terminology}{5}{section.3.1}}
+\newlabel{general:terminology}{{3.1}{5}{Terminology\relax }{section.3.1}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {3.2}Storing Authentication Information}{5}{section.3.2}}
+\newlabel{general:storing-authentication-information}{{3.2}{5}{Storing Authentication Information\relax }{section.3.2}{}}
+\newlabel{general:bucket-pw}{{3.2}{5}{Storing Authentication Information\relax }{section.3.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {3.3}On Backend Reliability}{6}{section.3.3}}
+\newlabel{general:backend-reliability}{{3.3}{6}{On Backend Reliability\relax }{section.3.3}{}}
+\newlabel{general:on-backend-reliability}{{3.3}{6}{On Backend Reliability\relax }{section.3.3}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {4}Storage Backends}{9}{chapter.4}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{mkfs::doc}{{4}{9}{File System Creation\relax }{chapter.4}{}}
-\newlabel{mkfs:file-system-creation}{{4}{9}{File System Creation\relax }{chapter.4}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {5}Managing Buckets}{11}{chapter.5}}
+\newlabel{backends:id1}{{4}{9}{Storage Backends\relax }{chapter.4}{}}
+\newlabel{backends::doc}{{4}{9}{Storage Backends\relax }{chapter.4}{}}
+\newlabel{backends:storage-backends}{{4}{9}{Storage Backends\relax }{chapter.4}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {4.1}Google Storage}{9}{section.4.1}}
+\newlabel{backends:google-storage}{{4.1}{9}{Google Storage\relax }{section.4.1}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {4.2}Amazon S3}{9}{section.4.2}}
+\newlabel{backends:amazon-s3}{{4.2}{9}{Amazon S3\relax }{section.4.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.2.1}Reduced Redundancy Storage (RRS)}{10}{subsection.4.2.1}}
+\newlabel{backends:reduced-redundancy-storage-rrs}{{4.2.1}{10}{Reduced Redundancy Storage (RRS)\relax }{subsection.4.2.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.2.2}Potential issues when using the US Standard storage region}{10}{subsection.4.2.2}}
+\newlabel{backends:potential-issues-when-using-the-us-standard-storage-region}{{4.2.2}{10}{Potential issues when using the US Standard storage region\relax }{subsection.4.2.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {4.3}S3 compatible}{10}{section.4.3}}
+\newlabel{backends:s3-compatible}{{4.3}{10}{S3 compatible\relax }{section.4.3}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {4.4}Local}{11}{section.4.4}}
+\newlabel{backends:local}{{4.4}{11}{Local\relax }{section.4.4}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {4.5}SSH/SFTP}{11}{section.4.5}}
+\newlabel{backends:ssh-sftp}{{4.5}{11}{SSH/SFTP\relax }{section.4.5}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {5}File System Creation}{13}{chapter.5}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{adm::doc}{{5}{11}{Managing Buckets\relax }{chapter.5}{}}
-\newlabel{adm:managing-buckets}{{5}{11}{Managing Buckets\relax }{chapter.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {5.1}Changing the Passphrase}{11}{section.5.1}}
-\newlabel{adm:changing-the-passphrase}{{5.1}{11}{Changing the Passphrase\relax }{section.5.1}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {5.2}Upgrading the file system}{11}{section.5.2}}
-\newlabel{adm:upgrading-the-file-system}{{5.2}{11}{Upgrading the file system\relax }{section.5.2}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {5.3}Deleting a file system}{12}{section.5.3}}
-\newlabel{adm:deleting-a-file-system}{{5.3}{12}{Deleting a file system\relax }{section.5.3}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {5.4}Restoring Metadata Backups}{12}{section.5.4}}
-\newlabel{adm:restoring-metadata-backups}{{5.4}{12}{Restoring Metadata Backups\relax }{section.5.4}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {6}Mounting}{13}{chapter.6}}
+\newlabel{mkfs::doc}{{5}{13}{File System Creation\relax }{chapter.5}{}}
+\newlabel{mkfs:file-system-creation}{{5}{13}{File System Creation\relax }{chapter.5}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {6}Managing Buckets}{15}{chapter.6}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{mount:mounting}{{6}{13}{Mounting\relax }{chapter.6}{}}
-\newlabel{mount::doc}{{6}{13}{Mounting\relax }{chapter.6}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {6.1}Storing Encryption Passwords}{14}{section.6.1}}
-\newlabel{mount:bucket-pw}{{6.1}{14}{Storing Encryption Passwords\relax }{section.6.1}{}}
-\newlabel{mount:storing-encryption-passwords}{{6.1}{14}{Storing Encryption Passwords\relax }{section.6.1}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {6.2}Compression Algorithms}{14}{section.6.2}}
-\newlabel{mount:compression-algorithms}{{6.2}{14}{Compression Algorithms\relax }{section.6.2}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {6.3}Parallel Compression}{15}{section.6.3}}
-\newlabel{mount:parallel-compression}{{6.3}{15}{Parallel Compression\relax }{section.6.3}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {6.4}Notes about Caching}{15}{section.6.4}}
-\newlabel{mount:notes-about-caching}{{6.4}{15}{Notes about Caching\relax }{section.6.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {6.4.1}Maximum Number of Cache Entries}{15}{subsection.6.4.1}}
-\newlabel{mount:maximum-number-of-cache-entries}{{6.4.1}{15}{Maximum Number of Cache Entries\relax }{subsection.6.4.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {6.4.2}Cache Flushing and Expiration}{15}{subsection.6.4.2}}
-\newlabel{mount:cache-flushing-and-expiration}{{6.4.2}{15}{Cache Flushing and Expiration\relax }{subsection.6.4.2}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {6.5}Automatic Mounting}{15}{section.6.5}}
-\newlabel{mount:automatic-mounting}{{6.5}{15}{Automatic Mounting\relax }{section.6.5}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {7}Advanced S3QL Features}{17}{chapter.7}}
+\newlabel{adm::doc}{{6}{15}{Managing Buckets\relax }{chapter.6}{}}
+\newlabel{adm:managing-buckets}{{6}{15}{Managing Buckets\relax }{chapter.6}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {6.1}Changing the Passphrase}{15}{section.6.1}}
+\newlabel{adm:changing-the-passphrase}{{6.1}{15}{Changing the Passphrase\relax }{section.6.1}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {6.2}Upgrading the file system}{15}{section.6.2}}
+\newlabel{adm:upgrading-the-file-system}{{6.2}{15}{Upgrading the file system\relax }{section.6.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {6.3}Deleting a file system}{16}{section.6.3}}
+\newlabel{adm:deleting-a-file-system}{{6.3}{16}{Deleting a file system\relax }{section.6.3}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {6.4}Restoring Metadata Backups}{16}{section.6.4}}
+\newlabel{adm:restoring-metadata-backups}{{6.4}{16}{Restoring Metadata Backups\relax }{section.6.4}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {7}Mounting}{17}{chapter.7}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{special:advanced-s3ql-features}{{7}{17}{Advanced S3QL Features\relax }{chapter.7}{}}
-\newlabel{special::doc}{{7}{17}{Advanced S3QL Features\relax }{chapter.7}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {7.1}Snapshotting and Copy-on-Write}{17}{section.7.1}}
-\newlabel{special:snapshotting-and-copy-on-write}{{7.1}{17}{Snapshotting and Copy-on-Write\relax }{section.7.1}{}}
-\newlabel{special:s3qlcp}{{7.1}{17}{Snapshotting and Copy-on-Write\relax }{section.7.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {7.1.1}Snapshotting vs Hardlinking}{17}{subsection.7.1.1}}
-\newlabel{special:snapshotting-vs-hardlinking}{{7.1.1}{17}{Snapshotting vs Hardlinking\relax }{subsection.7.1.1}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {7.2}Getting Statistics}{18}{section.7.2}}
-\newlabel{special:s3qlstat}{{7.2}{18}{Getting Statistics\relax }{section.7.2}{}}
-\newlabel{special:getting-statistics}{{7.2}{18}{Getting Statistics\relax }{section.7.2}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {7.3}Immutable Trees}{18}{section.7.3}}
-\newlabel{special:immutable-trees}{{7.3}{18}{Immutable Trees\relax }{section.7.3}{}}
-\newlabel{special:s3qllock}{{7.3}{18}{Immutable Trees\relax }{section.7.3}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {7.4}Fast Recursive Removal}{19}{section.7.4}}
-\newlabel{special:s3qlrm}{{7.4}{19}{Fast Recursive Removal\relax }{section.7.4}{}}
-\newlabel{special:fast-recursive-removal}{{7.4}{19}{Fast Recursive Removal\relax }{section.7.4}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {7.5}Runtime Configuration}{19}{section.7.5}}
-\newlabel{special:runtime-configuration}{{7.5}{19}{Runtime Configuration\relax }{section.7.5}{}}
-\newlabel{special:s3qlctrl}{{7.5}{19}{Runtime Configuration\relax }{section.7.5}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {8}Unmounting}{21}{chapter.8}}
+\newlabel{mount:mounting}{{7}{17}{Mounting\relax }{chapter.7}{}}
+\newlabel{mount::doc}{{7}{17}{Mounting\relax }{chapter.7}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {7.1}Compression Algorithms}{18}{section.7.1}}
+\newlabel{mount:compression-algorithms}{{7.1}{18}{Compression Algorithms\relax }{section.7.1}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {7.2}Parallel Compression}{18}{section.7.2}}
+\newlabel{mount:parallel-compression}{{7.2}{18}{Parallel Compression\relax }{section.7.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {7.3}Notes about Caching}{19}{section.7.3}}
+\newlabel{mount:notes-about-caching}{{7.3}{19}{Notes about Caching\relax }{section.7.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {7.3.1}Maximum Number of Cache Entries}{19}{subsection.7.3.1}}
+\newlabel{mount:maximum-number-of-cache-entries}{{7.3.1}{19}{Maximum Number of Cache Entries\relax }{subsection.7.3.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {7.3.2}Cache Flushing and Expiration}{19}{subsection.7.3.2}}
+\newlabel{mount:cache-flushing-and-expiration}{{7.3.2}{19}{Cache Flushing and Expiration\relax }{subsection.7.3.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {7.4}Automatic Mounting}{19}{section.7.4}}
+\newlabel{mount:automatic-mounting}{{7.4}{19}{Automatic Mounting\relax }{section.7.4}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {8}Advanced S3QL Features}{21}{chapter.8}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{umount::doc}{{8}{21}{Unmounting\relax }{chapter.8}{}}
-\newlabel{umount:unmounting}{{8}{21}{Unmounting\relax }{chapter.8}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {9}Checking for Errors}{23}{chapter.9}}
+\newlabel{special:advanced-s3ql-features}{{8}{21}{Advanced S3QL Features\relax }{chapter.8}{}}
+\newlabel{special::doc}{{8}{21}{Advanced S3QL Features\relax }{chapter.8}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {8.1}Snapshotting and Copy-on-Write}{21}{section.8.1}}
+\newlabel{special:snapshotting-and-copy-on-write}{{8.1}{21}{Snapshotting and Copy-on-Write\relax }{section.8.1}{}}
+\newlabel{special:s3qlcp}{{8.1}{21}{Snapshotting and Copy-on-Write\relax }{section.8.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {8.1.1}Snapshotting vs Hardlinking}{21}{subsection.8.1.1}}
+\newlabel{special:snapshotting-vs-hardlinking}{{8.1.1}{21}{Snapshotting vs Hardlinking\relax }{subsection.8.1.1}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {8.2}Getting Statistics}{22}{section.8.2}}
+\newlabel{special:s3qlstat}{{8.2}{22}{Getting Statistics\relax }{section.8.2}{}}
+\newlabel{special:getting-statistics}{{8.2}{22}{Getting Statistics\relax }{section.8.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {8.3}Immutable Trees}{22}{section.8.3}}
+\newlabel{special:immutable-trees}{{8.3}{22}{Immutable Trees\relax }{section.8.3}{}}
+\newlabel{special:s3qllock}{{8.3}{22}{Immutable Trees\relax }{section.8.3}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {8.4}Fast Recursive Removal}{23}{section.8.4}}
+\newlabel{special:s3qlrm}{{8.4}{23}{Fast Recursive Removal\relax }{section.8.4}{}}
+\newlabel{special:fast-recursive-removal}{{8.4}{23}{Fast Recursive Removal\relax }{section.8.4}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {8.5}Runtime Configuration}{23}{section.8.5}}
+\newlabel{special:runtime-configuration}{{8.5}{23}{Runtime Configuration\relax }{section.8.5}{}}
+\newlabel{special:s3qlctrl}{{8.5}{23}{Runtime Configuration\relax }{section.8.5}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {9}Unmounting}{25}{chapter.9}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{fsck:checking-for-errors}{{9}{23}{Checking for Errors\relax }{chapter.9}{}}
-\newlabel{fsck::doc}{{9}{23}{Checking for Errors\relax }{chapter.9}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {10}Contributed Programs}{25}{chapter.10}}
+\newlabel{umount::doc}{{9}{25}{Unmounting\relax }{chapter.9}{}}
+\newlabel{umount:unmounting}{{9}{25}{Unmounting\relax }{chapter.9}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {10}Checking for Errors}{27}{chapter.10}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{contrib:contributed-programs}{{10}{25}{Contributed Programs\relax }{chapter.10}{}}
-\newlabel{contrib::doc}{{10}{25}{Contributed Programs\relax }{chapter.10}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {10.1}benchmark.py}{25}{section.10.1}}
-\newlabel{contrib:benchmark-py}{{10.1}{25}{benchmark.py\relax }{section.10.1}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {10.2}s3\_copy.py}{25}{section.10.2}}
-\newlabel{contrib:s3-copy-py}{{10.2}{25}{s3\_copy.py\relax }{section.10.2}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {10.3}pcp.py}{25}{section.10.3}}
-\newlabel{contrib:pcp-py}{{10.3}{25}{pcp.py\relax }{section.10.3}{}}
-\newlabel{contrib:pcp}{{10.3}{25}{pcp.py\relax }{section.10.3}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {10.4}s3\_backup.sh}{25}{section.10.4}}
-\newlabel{contrib:s3-backup-sh}{{10.4}{25}{s3\_backup.sh\relax }{section.10.4}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {10.5}expire\_backups.py}{26}{section.10.5}}
-\newlabel{contrib:expire-backups-py}{{10.5}{26}{expire\_backups.py\relax }{section.10.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {10.6}s3ql.conf}{27}{section.10.6}}
-\newlabel{contrib:s3ql-conf}{{10.6}{27}{s3ql.conf\relax }{section.10.6}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {11}Tips \& Tricks}{29}{chapter.11}}
+\newlabel{fsck:checking-for-errors}{{10}{27}{Checking for Errors\relax }{chapter.10}{}}
+\newlabel{fsck::doc}{{10}{27}{Checking for Errors\relax }{chapter.10}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {11}Contributed Programs}{29}{chapter.11}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{tips:tips-tricks}{{11}{29}{Tips \& Tricks\relax }{chapter.11}{}}
-\newlabel{tips::doc}{{11}{29}{Tips \& Tricks\relax }{chapter.11}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {11.1}Permanently mounted backup file system}{29}{section.11.1}}
-\newlabel{tips:copy-performance}{{11.1}{29}{Permanently mounted backup file system\relax }{section.11.1}{}}
-\newlabel{tips:permanently-mounted-backup-file-system}{{11.1}{29}{Permanently mounted backup file system\relax }{section.11.1}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {11.2}Improving copy performance}{29}{section.11.2}}
-\newlabel{tips:improving-copy-performance}{{11.2}{29}{Improving copy performance\relax }{section.11.2}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {12}Known Issues}{31}{chapter.12}}
+\newlabel{contrib:contributed-programs}{{11}{29}{Contributed Programs\relax }{chapter.11}{}}
+\newlabel{contrib::doc}{{11}{29}{Contributed Programs\relax }{chapter.11}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {11.1}benchmark.py}{29}{section.11.1}}
+\newlabel{contrib:benchmark-py}{{11.1}{29}{benchmark.py\relax }{section.11.1}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {11.2}s3\_copy.py}{29}{section.11.2}}
+\newlabel{contrib:s3-copy-py}{{11.2}{29}{s3\_copy.py\relax }{section.11.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {11.3}pcp.py}{29}{section.11.3}}
+\newlabel{contrib:pcp-py}{{11.3}{29}{pcp.py\relax }{section.11.3}{}}
+\newlabel{contrib:pcp}{{11.3}{29}{pcp.py\relax }{section.11.3}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {11.4}s3\_backup.sh}{29}{section.11.4}}
+\newlabel{contrib:s3-backup-sh}{{11.4}{29}{s3\_backup.sh\relax }{section.11.4}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {11.5}expire\_backups.py}{30}{section.11.5}}
+\newlabel{contrib:expire-backups-py}{{11.5}{30}{expire\_backups.py\relax }{section.11.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {11.6}s3ql.conf}{31}{section.11.6}}
+\newlabel{contrib:s3ql-conf}{{11.6}{31}{s3ql.conf\relax }{section.11.6}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {12}Tips \& Tricks}{33}{chapter.12}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{issues:known-issues}{{12}{31}{Known Issues\relax }{chapter.12}{}}
-\newlabel{issues::doc}{{12}{31}{Known Issues\relax }{chapter.12}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {13}Manpages}{33}{chapter.13}}
+\newlabel{tips:tips-tricks}{{12}{33}{Tips \& Tricks\relax }{chapter.12}{}}
+\newlabel{tips::doc}{{12}{33}{Tips \& Tricks\relax }{chapter.12}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {12.1}SSH Backend}{33}{section.12.1}}
+\newlabel{tips:ssh-tipp}{{12.1}{33}{SSH Backend\relax }{section.12.1}{}}
+\newlabel{tips:ssh-backend}{{12.1}{33}{SSH Backend\relax }{section.12.1}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {12.2}Permanently mounted backup file system}{33}{section.12.2}}
+\newlabel{tips:permanently-mounted-backup-file-system}{{12.2}{33}{Permanently mounted backup file system\relax }{section.12.2}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {12.3}Improving copy performance}{33}{section.12.3}}
+\newlabel{tips:copy-performance}{{12.3}{33}{Improving copy performance\relax }{section.12.3}{}}
+\newlabel{tips:improving-copy-performance}{{12.3}{33}{Improving copy performance\relax }{section.12.3}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {13}Known Issues}{35}{chapter.13}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{man/index:manpages}{{13}{33}{Manpages\relax }{chapter.13}{}}
-\newlabel{man/index::doc}{{13}{33}{Manpages\relax }{chapter.13}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.1}The \textbf {mkfs.s3ql} command}{33}{section.13.1}}
-\newlabel{man/mkfs:the-mkfs-s3ql-command}{{13.1}{33}{The \textbf {mkfs.s3ql} command\relax }{section.13.1}{}}
-\newlabel{man/mkfs::doc}{{13.1}{33}{The \textbf {mkfs.s3ql} command\relax }{section.13.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.1.1}Synopsis}{33}{subsection.13.1.1}}
-\newlabel{man/mkfs:synopsis}{{13.1.1}{33}{Synopsis\relax }{subsection.13.1.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.1.2}Description}{33}{subsection.13.1.2}}
-\newlabel{man/mkfs:description}{{13.1.2}{33}{Description\relax }{subsection.13.1.2}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Amazon S3}{33}{subsubsection*.3}}
-\newlabel{man/mkfs:amazon-s3}{{13.1.2}{33}{Amazon S3\relax }{subsubsection*.3}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Local}{33}{subsubsection*.4}}
-\newlabel{man/mkfs:local}{{13.1.2}{33}{Local\relax }{subsubsection*.4}{}}
-\@writefile{toc}{\contentsline {subsubsection}{SFTP}{33}{subsubsection*.5}}
-\newlabel{man/mkfs:sftp}{{13.1.2}{33}{SFTP\relax }{subsubsection*.5}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.1.3}Options}{34}{subsection.13.1.3}}
-\newlabel{man/mkfs:options}{{13.1.3}{34}{Options\relax }{subsection.13.1.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.1.4}Files}{34}{subsection.13.1.4}}
-\newlabel{man/mkfs:files}{{13.1.4}{34}{Files\relax }{subsection.13.1.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.1.5}Exit Status}{34}{subsection.13.1.5}}
-\newlabel{man/mkfs:exit-status}{{13.1.5}{34}{Exit Status\relax }{subsection.13.1.5}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.1.6}See Also}{34}{subsection.13.1.6}}
-\newlabel{man/mkfs:see-also}{{13.1.6}{34}{See Also\relax }{subsection.13.1.6}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.2}The \textbf {s3qladm} command}{34}{section.13.2}}
-\newlabel{man/adm::doc}{{13.2}{34}{The \textbf {s3qladm} command\relax }{section.13.2}{}}
-\newlabel{man/adm:the-s3qladm-command}{{13.2}{34}{The \textbf {s3qladm} command\relax }{section.13.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.2.1}Synopsis}{34}{subsection.13.2.1}}
-\newlabel{man/adm:synopsis}{{13.2.1}{34}{Synopsis\relax }{subsection.13.2.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.2.2}Description}{35}{subsection.13.2.2}}
-\newlabel{man/adm:description}{{13.2.2}{35}{Description\relax }{subsection.13.2.2}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Amazon S3}{35}{subsubsection*.6}}
-\newlabel{man/adm:amazon-s3}{{13.2.2}{35}{Amazon S3\relax }{subsubsection*.6}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Local}{35}{subsubsection*.7}}
-\newlabel{man/adm:local}{{13.2.2}{35}{Local\relax }{subsubsection*.7}{}}
-\@writefile{toc}{\contentsline {subsubsection}{SFTP}{35}{subsubsection*.8}}
-\newlabel{man/adm:sftp}{{13.2.2}{35}{SFTP\relax }{subsubsection*.8}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.2.3}Options}{35}{subsection.13.2.3}}
-\newlabel{man/adm:options}{{13.2.3}{35}{Options\relax }{subsection.13.2.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.2.4}Actions}{35}{subsection.13.2.4}}
-\newlabel{man/adm:actions}{{13.2.4}{35}{Actions\relax }{subsection.13.2.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.2.5}Files}{36}{subsection.13.2.5}}
-\newlabel{man/adm:files}{{13.2.5}{36}{Files\relax }{subsection.13.2.5}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.2.6}Exit Status}{36}{subsection.13.2.6}}
-\newlabel{man/adm:exit-status}{{13.2.6}{36}{Exit Status\relax }{subsection.13.2.6}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.2.7}See Also}{36}{subsection.13.2.7}}
-\newlabel{man/adm:see-also}{{13.2.7}{36}{See Also\relax }{subsection.13.2.7}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.3}The \textbf {mount.s3ql} command}{36}{section.13.3}}
-\newlabel{man/mount::doc}{{13.3}{36}{The \textbf {mount.s3ql} command\relax }{section.13.3}{}}
-\newlabel{man/mount:the-mount-s3ql-command}{{13.3}{36}{The \textbf {mount.s3ql} command\relax }{section.13.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.3.1}Synopsis}{36}{subsection.13.3.1}}
-\newlabel{man/mount:synopsis}{{13.3.1}{36}{Synopsis\relax }{subsection.13.3.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.3.2}Description}{36}{subsection.13.3.2}}
-\newlabel{man/mount:description}{{13.3.2}{36}{Description\relax }{subsection.13.3.2}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Amazon S3}{36}{subsubsection*.9}}
-\newlabel{man/mount:amazon-s3}{{13.3.2}{36}{Amazon S3\relax }{subsubsection*.9}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Local}{36}{subsubsection*.10}}
-\newlabel{man/mount:local}{{13.3.2}{36}{Local\relax }{subsubsection*.10}{}}
-\@writefile{toc}{\contentsline {subsubsection}{SFTP}{36}{subsubsection*.11}}
-\newlabel{man/mount:sftp}{{13.3.2}{36}{SFTP\relax }{subsubsection*.11}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.3.3}Options}{37}{subsection.13.3.3}}
-\newlabel{man/mount:options}{{13.3.3}{37}{Options\relax }{subsection.13.3.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.3.4}Files}{38}{subsection.13.3.4}}
-\newlabel{man/mount:files}{{13.3.4}{38}{Files\relax }{subsection.13.3.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.3.5}Exit Status}{38}{subsection.13.3.5}}
-\newlabel{man/mount:exit-status}{{13.3.5}{38}{Exit Status\relax }{subsection.13.3.5}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.3.6}See Also}{38}{subsection.13.3.6}}
-\newlabel{man/mount:see-also}{{13.3.6}{38}{See Also\relax }{subsection.13.3.6}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.4}The \textbf {s3qlstat} command}{38}{section.13.4}}
-\newlabel{man/stat:the-s3qlstat-command}{{13.4}{38}{The \textbf {s3qlstat} command\relax }{section.13.4}{}}
-\newlabel{man/stat::doc}{{13.4}{38}{The \textbf {s3qlstat} command\relax }{section.13.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.4.1}Synopsis}{38}{subsection.13.4.1}}
-\newlabel{man/stat:synopsis}{{13.4.1}{38}{Synopsis\relax }{subsection.13.4.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.4.2}Description}{38}{subsection.13.4.2}}
-\newlabel{man/stat:description}{{13.4.2}{38}{Description\relax }{subsection.13.4.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.4.3}Options}{38}{subsection.13.4.3}}
-\newlabel{man/stat:options}{{13.4.3}{38}{Options\relax }{subsection.13.4.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.4.4}Exit Status}{38}{subsection.13.4.4}}
-\newlabel{man/stat:exit-status}{{13.4.4}{38}{Exit Status\relax }{subsection.13.4.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.4.5}See Also}{38}{subsection.13.4.5}}
-\newlabel{man/stat:see-also}{{13.4.5}{38}{See Also\relax }{subsection.13.4.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.5}The \textbf {s3qlctrl} command}{39}{section.13.5}}
-\newlabel{man/ctrl:the-s3qlctrl-command}{{13.5}{39}{The \textbf {s3qlctrl} command\relax }{section.13.5}{}}
-\newlabel{man/ctrl::doc}{{13.5}{39}{The \textbf {s3qlctrl} command\relax }{section.13.5}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.5.1}Synopsis}{39}{subsection.13.5.1}}
-\newlabel{man/ctrl:synopsis}{{13.5.1}{39}{Synopsis\relax }{subsection.13.5.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.5.2}Description}{39}{subsection.13.5.2}}
-\newlabel{man/ctrl:description}{{13.5.2}{39}{Description\relax }{subsection.13.5.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.5.3}Options}{39}{subsection.13.5.3}}
-\newlabel{man/ctrl:options}{{13.5.3}{39}{Options\relax }{subsection.13.5.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.5.4}Exit Status}{39}{subsection.13.5.4}}
-\newlabel{man/ctrl:exit-status}{{13.5.4}{39}{Exit Status\relax }{subsection.13.5.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.5.5}See Also}{39}{subsection.13.5.5}}
-\newlabel{man/ctrl:see-also}{{13.5.5}{39}{See Also\relax }{subsection.13.5.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.6}The \textbf {s3qlcp} command}{40}{section.13.6}}
-\newlabel{man/cp:the-s3qlcp-command}{{13.6}{40}{The \textbf {s3qlcp} command\relax }{section.13.6}{}}
-\newlabel{man/cp::doc}{{13.6}{40}{The \textbf {s3qlcp} command\relax }{section.13.6}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.6.1}Synopsis}{40}{subsection.13.6.1}}
-\newlabel{man/cp:synopsis}{{13.6.1}{40}{Synopsis\relax }{subsection.13.6.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.6.2}Description}{40}{subsection.13.6.2}}
-\newlabel{man/cp:description}{{13.6.2}{40}{Description\relax }{subsection.13.6.2}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Snapshotting vs Hardlinking}{40}{subsubsection*.12}}
-\newlabel{man/cp:snapshotting-vs-hardlinking}{{13.6.2}{40}{Snapshotting vs Hardlinking\relax }{subsubsection*.12}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.6.3}Options}{40}{subsection.13.6.3}}
-\newlabel{man/cp:options}{{13.6.3}{40}{Options\relax }{subsection.13.6.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.6.4}Exit Status}{41}{subsection.13.6.4}}
-\newlabel{man/cp:exit-status}{{13.6.4}{41}{Exit Status\relax }{subsection.13.6.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.6.5}See Also}{41}{subsection.13.6.5}}
-\newlabel{man/cp:see-also}{{13.6.5}{41}{See Also\relax }{subsection.13.6.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.7}The \textbf {s3qlrm} command}{41}{section.13.7}}
-\newlabel{man/rm::doc}{{13.7}{41}{The \textbf {s3qlrm} command\relax }{section.13.7}{}}
-\newlabel{man/rm:the-s3qlrm-command}{{13.7}{41}{The \textbf {s3qlrm} command\relax }{section.13.7}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.7.1}Synopsis}{41}{subsection.13.7.1}}
-\newlabel{man/rm:synopsis}{{13.7.1}{41}{Synopsis\relax }{subsection.13.7.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.7.2}Description}{41}{subsection.13.7.2}}
-\newlabel{man/rm:description}{{13.7.2}{41}{Description\relax }{subsection.13.7.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.7.3}Options}{41}{subsection.13.7.3}}
-\newlabel{man/rm:options}{{13.7.3}{41}{Options\relax }{subsection.13.7.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.7.4}Exit Status}{41}{subsection.13.7.4}}
-\newlabel{man/rm:exit-status}{{13.7.4}{41}{Exit Status\relax }{subsection.13.7.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.7.5}See Also}{41}{subsection.13.7.5}}
-\newlabel{man/rm:see-also}{{13.7.5}{41}{See Also\relax }{subsection.13.7.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.8}The \textbf {s3qllock} command}{42}{section.13.8}}
-\newlabel{man/lock:the-s3qllock-command}{{13.8}{42}{The \textbf {s3qllock} command\relax }{section.13.8}{}}
-\newlabel{man/lock::doc}{{13.8}{42}{The \textbf {s3qllock} command\relax }{section.13.8}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.8.1}Synopsis}{42}{subsection.13.8.1}}
-\newlabel{man/lock:synopsis}{{13.8.1}{42}{Synopsis\relax }{subsection.13.8.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.8.2}Description}{42}{subsection.13.8.2}}
-\newlabel{man/lock:description}{{13.8.2}{42}{Description\relax }{subsection.13.8.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.8.3}Rationale}{42}{subsection.13.8.3}}
-\newlabel{man/lock:rationale}{{13.8.3}{42}{Rationale\relax }{subsection.13.8.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.8.4}Options}{42}{subsection.13.8.4}}
-\newlabel{man/lock:options}{{13.8.4}{42}{Options\relax }{subsection.13.8.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.8.5}Exit Status}{42}{subsection.13.8.5}}
-\newlabel{man/lock:exit-status}{{13.8.5}{42}{Exit Status\relax }{subsection.13.8.5}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.8.6}See Also}{43}{subsection.13.8.6}}
-\newlabel{man/lock:see-also}{{13.8.6}{43}{See Also\relax }{subsection.13.8.6}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.9}The \textbf {umount.s3ql} command}{43}{section.13.9}}
-\newlabel{man/umount::doc}{{13.9}{43}{The \textbf {umount.s3ql} command\relax }{section.13.9}{}}
-\newlabel{man/umount:the-umount-s3ql-command}{{13.9}{43}{The \textbf {umount.s3ql} command\relax }{section.13.9}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.9.1}Synopsis}{43}{subsection.13.9.1}}
-\newlabel{man/umount:synopsis}{{13.9.1}{43}{Synopsis\relax }{subsection.13.9.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.9.2}Description}{43}{subsection.13.9.2}}
-\newlabel{man/umount:description}{{13.9.2}{43}{Description\relax }{subsection.13.9.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.9.3}Options}{43}{subsection.13.9.3}}
-\newlabel{man/umount:options}{{13.9.3}{43}{Options\relax }{subsection.13.9.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.9.4}Exit Status}{43}{subsection.13.9.4}}
-\newlabel{man/umount:exit-status}{{13.9.4}{43}{Exit Status\relax }{subsection.13.9.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.9.5}See Also}{43}{subsection.13.9.5}}
-\newlabel{man/umount:see-also}{{13.9.5}{43}{See Also\relax }{subsection.13.9.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.10}The \textbf {fsck.s3ql} command}{44}{section.13.10}}
-\newlabel{man/fsck::doc}{{13.10}{44}{The \textbf {fsck.s3ql} command\relax }{section.13.10}{}}
-\newlabel{man/fsck:the-fsck-s3ql-command}{{13.10}{44}{The \textbf {fsck.s3ql} command\relax }{section.13.10}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.10.1}Synopsis}{44}{subsection.13.10.1}}
-\newlabel{man/fsck:synopsis}{{13.10.1}{44}{Synopsis\relax }{subsection.13.10.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.10.2}Description}{44}{subsection.13.10.2}}
-\newlabel{man/fsck:description}{{13.10.2}{44}{Description\relax }{subsection.13.10.2}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Amazon S3}{44}{subsubsection*.13}}
-\newlabel{man/fsck:amazon-s3}{{13.10.2}{44}{Amazon S3\relax }{subsubsection*.13}{}}
-\@writefile{toc}{\contentsline {subsubsection}{Local}{44}{subsubsection*.14}}
-\newlabel{man/fsck:local}{{13.10.2}{44}{Local\relax }{subsubsection*.14}{}}
-\@writefile{toc}{\contentsline {subsubsection}{SFTP}{44}{subsubsection*.15}}
-\newlabel{man/fsck:sftp}{{13.10.2}{44}{SFTP\relax }{subsubsection*.15}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.10.3}Options}{44}{subsection.13.10.3}}
-\newlabel{man/fsck:options}{{13.10.3}{44}{Options\relax }{subsection.13.10.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.10.4}Files}{45}{subsection.13.10.4}}
-\newlabel{man/fsck:files}{{13.10.4}{45}{Files\relax }{subsection.13.10.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.10.5}Exit Status}{45}{subsection.13.10.5}}
-\newlabel{man/fsck:exit-status}{{13.10.5}{45}{Exit Status\relax }{subsection.13.10.5}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.10.6}See Also}{45}{subsection.13.10.6}}
-\newlabel{man/fsck:see-also}{{13.10.6}{45}{See Also\relax }{subsection.13.10.6}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.11}The \textbf {pcp} command}{45}{section.13.11}}
-\newlabel{man/pcp:the-pcp-command}{{13.11}{45}{The \textbf {pcp} command\relax }{section.13.11}{}}
-\newlabel{man/pcp::doc}{{13.11}{45}{The \textbf {pcp} command\relax }{section.13.11}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.11.1}Synopsis}{45}{subsection.13.11.1}}
-\newlabel{man/pcp:synopsis}{{13.11.1}{45}{Synopsis\relax }{subsection.13.11.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.11.2}Description}{45}{subsection.13.11.2}}
-\newlabel{man/pcp:description}{{13.11.2}{45}{Description\relax }{subsection.13.11.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.11.3}Options}{45}{subsection.13.11.3}}
-\newlabel{man/pcp:options}{{13.11.3}{45}{Options\relax }{subsection.13.11.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.11.4}Exit Status}{45}{subsection.13.11.4}}
-\newlabel{man/pcp:exit-status}{{13.11.4}{45}{Exit Status\relax }{subsection.13.11.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.11.5}See Also}{45}{subsection.13.11.5}}
-\newlabel{man/pcp:see-also}{{13.11.5}{45}{See Also\relax }{subsection.13.11.5}{}}
-\@writefile{toc}{\contentsline {section}{\numberline {13.12}The \textbf {expire\_backups} command}{46}{section.13.12}}
-\newlabel{man/expire_backups::doc}{{13.12}{46}{The \textbf {expire\_backups} command\relax }{section.13.12}{}}
-\newlabel{man/expire_backups:the-expire-backups-command}{{13.12}{46}{The \textbf {expire\_backups} command\relax }{section.13.12}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.12.1}Synopsis}{46}{subsection.13.12.1}}
-\newlabel{man/expire_backups:synopsis}{{13.12.1}{46}{Synopsis\relax }{subsection.13.12.1}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.12.2}Description}{46}{subsection.13.12.2}}
-\newlabel{man/expire_backups:description}{{13.12.2}{46}{Description\relax }{subsection.13.12.2}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.12.3}Options}{47}{subsection.13.12.3}}
-\newlabel{man/expire_backups:options}{{13.12.3}{47}{Options\relax }{subsection.13.12.3}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.12.4}Exit Status}{47}{subsection.13.12.4}}
-\newlabel{man/expire_backups:exit-status}{{13.12.4}{47}{Exit Status\relax }{subsection.13.12.4}{}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {13.12.5}See Also}{47}{subsection.13.12.5}}
-\newlabel{man/expire_backups:see-also}{{13.12.5}{47}{See Also\relax }{subsection.13.12.5}{}}
-\@writefile{toc}{\contentsline {chapter}{\numberline {14}Further Resources / Getting Help}{49}{chapter.14}}
+\newlabel{issues:known-issues}{{13}{35}{Known Issues\relax }{chapter.13}{}}
+\newlabel{issues::doc}{{13}{35}{Known Issues\relax }{chapter.13}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {14}Manpages}{37}{chapter.14}}
\@writefile{lof}{\addvspace {10\p@ }}
\@writefile{lot}{\addvspace {10\p@ }}
-\newlabel{resources::doc}{{14}{49}{Further Resources / Getting Help\relax }{chapter.14}{}}
-\newlabel{resources:further-resources-getting-help}{{14}{49}{Further Resources / Getting Help\relax }{chapter.14}{}}
-\newlabel{resources:resources}{{14}{49}{Further Resources / Getting Help\relax }{chapter.14}{}}
+\newlabel{man/index:manpages}{{14}{37}{Manpages\relax }{chapter.14}{}}
+\newlabel{man/index::doc}{{14}{37}{Manpages\relax }{chapter.14}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.1}The \textbf {mkfs.s3ql} command}{37}{section.14.1}}
+\newlabel{man/mkfs:the-mkfs-s3ql-command}{{14.1}{37}{The \textbf {mkfs.s3ql} command\relax }{section.14.1}{}}
+\newlabel{man/mkfs::doc}{{14.1}{37}{The \textbf {mkfs.s3ql} command\relax }{section.14.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.1.1}Synopsis}{37}{subsection.14.1.1}}
+\newlabel{man/mkfs:synopsis}{{14.1.1}{37}{Synopsis\relax }{subsection.14.1.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.1.2}Description}{37}{subsection.14.1.2}}
+\newlabel{man/mkfs:description}{{14.1.2}{37}{Description\relax }{subsection.14.1.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.1.3}Options}{37}{subsection.14.1.3}}
+\newlabel{man/mkfs:options}{{14.1.3}{37}{Options\relax }{subsection.14.1.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.1.4}Exit Status}{38}{subsection.14.1.4}}
+\newlabel{man/mkfs:exit-status}{{14.1.4}{38}{Exit Status\relax }{subsection.14.1.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.1.5}See Also}{38}{subsection.14.1.5}}
+\newlabel{man/mkfs:see-also}{{14.1.5}{38}{See Also\relax }{subsection.14.1.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.2}The \textbf {s3qladm} command}{38}{section.14.2}}
+\newlabel{man/adm::doc}{{14.2}{38}{The \textbf {s3qladm} command\relax }{section.14.2}{}}
+\newlabel{man/adm:the-s3qladm-command}{{14.2}{38}{The \textbf {s3qladm} command\relax }{section.14.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.2.1}Synopsis}{38}{subsection.14.2.1}}
+\newlabel{man/adm:synopsis}{{14.2.1}{38}{Synopsis\relax }{subsection.14.2.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.2.2}Description}{38}{subsection.14.2.2}}
+\newlabel{man/adm:description}{{14.2.2}{38}{Description\relax }{subsection.14.2.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.2.3}Options}{38}{subsection.14.2.3}}
+\newlabel{man/adm:options}{{14.2.3}{38}{Options\relax }{subsection.14.2.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.2.4}Actions}{39}{subsection.14.2.4}}
+\newlabel{man/adm:actions}{{14.2.4}{39}{Actions\relax }{subsection.14.2.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.2.5}Exit Status}{39}{subsection.14.2.5}}
+\newlabel{man/adm:exit-status}{{14.2.5}{39}{Exit Status\relax }{subsection.14.2.5}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.2.6}See Also}{39}{subsection.14.2.6}}
+\newlabel{man/adm:see-also}{{14.2.6}{39}{See Also\relax }{subsection.14.2.6}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.3}The \textbf {mount.s3ql} command}{39}{section.14.3}}
+\newlabel{man/mount::doc}{{14.3}{39}{The \textbf {mount.s3ql} command\relax }{section.14.3}{}}
+\newlabel{man/mount:the-mount-s3ql-command}{{14.3}{39}{The \textbf {mount.s3ql} command\relax }{section.14.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.3.1}Synopsis}{39}{subsection.14.3.1}}
+\newlabel{man/mount:synopsis}{{14.3.1}{39}{Synopsis\relax }{subsection.14.3.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.3.2}Description}{39}{subsection.14.3.2}}
+\newlabel{man/mount:description}{{14.3.2}{39}{Description\relax }{subsection.14.3.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.3.3}Options}{39}{subsection.14.3.3}}
+\newlabel{man/mount:options}{{14.3.3}{39}{Options\relax }{subsection.14.3.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.3.4}Exit Status}{40}{subsection.14.3.4}}
+\newlabel{man/mount:exit-status}{{14.3.4}{40}{Exit Status\relax }{subsection.14.3.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.3.5}See Also}{40}{subsection.14.3.5}}
+\newlabel{man/mount:see-also}{{14.3.5}{40}{See Also\relax }{subsection.14.3.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.4}The \textbf {s3qlstat} command}{41}{section.14.4}}
+\newlabel{man/stat:the-s3qlstat-command}{{14.4}{41}{The \textbf {s3qlstat} command\relax }{section.14.4}{}}
+\newlabel{man/stat::doc}{{14.4}{41}{The \textbf {s3qlstat} command\relax }{section.14.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.4.1}Synopsis}{41}{subsection.14.4.1}}
+\newlabel{man/stat:synopsis}{{14.4.1}{41}{Synopsis\relax }{subsection.14.4.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.4.2}Description}{41}{subsection.14.4.2}}
+\newlabel{man/stat:description}{{14.4.2}{41}{Description\relax }{subsection.14.4.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.4.3}Options}{41}{subsection.14.4.3}}
+\newlabel{man/stat:options}{{14.4.3}{41}{Options\relax }{subsection.14.4.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.4.4}Exit Status}{41}{subsection.14.4.4}}
+\newlabel{man/stat:exit-status}{{14.4.4}{41}{Exit Status\relax }{subsection.14.4.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.4.5}See Also}{41}{subsection.14.4.5}}
+\newlabel{man/stat:see-also}{{14.4.5}{41}{See Also\relax }{subsection.14.4.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.5}The \textbf {s3qlctrl} command}{41}{section.14.5}}
+\newlabel{man/ctrl:the-s3qlctrl-command}{{14.5}{41}{The \textbf {s3qlctrl} command\relax }{section.14.5}{}}
+\newlabel{man/ctrl::doc}{{14.5}{41}{The \textbf {s3qlctrl} command\relax }{section.14.5}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.5.1}Synopsis}{41}{subsection.14.5.1}}
+\newlabel{man/ctrl:synopsis}{{14.5.1}{41}{Synopsis\relax }{subsection.14.5.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.5.2}Description}{41}{subsection.14.5.2}}
+\newlabel{man/ctrl:description}{{14.5.2}{41}{Description\relax }{subsection.14.5.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.5.3}Options}{42}{subsection.14.5.3}}
+\newlabel{man/ctrl:options}{{14.5.3}{42}{Options\relax }{subsection.14.5.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.5.4}Exit Status}{42}{subsection.14.5.4}}
+\newlabel{man/ctrl:exit-status}{{14.5.4}{42}{Exit Status\relax }{subsection.14.5.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.5.5}See Also}{42}{subsection.14.5.5}}
+\newlabel{man/ctrl:see-also}{{14.5.5}{42}{See Also\relax }{subsection.14.5.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.6}The \textbf {s3qlcp} command}{42}{section.14.6}}
+\newlabel{man/cp:the-s3qlcp-command}{{14.6}{42}{The \textbf {s3qlcp} command\relax }{section.14.6}{}}
+\newlabel{man/cp::doc}{{14.6}{42}{The \textbf {s3qlcp} command\relax }{section.14.6}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.6.1}Synopsis}{42}{subsection.14.6.1}}
+\newlabel{man/cp:synopsis}{{14.6.1}{42}{Synopsis\relax }{subsection.14.6.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.6.2}Description}{42}{subsection.14.6.2}}
+\newlabel{man/cp:description}{{14.6.2}{42}{Description\relax }{subsection.14.6.2}{}}
+\@writefile{toc}{\contentsline {subsubsection}{Snapshotting vs Hardlinking}{43}{subsubsection*.3}}
+\newlabel{man/cp:snapshotting-vs-hardlinking}{{14.6.2}{43}{Snapshotting vs Hardlinking\relax }{subsubsection*.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.6.3}Options}{43}{subsection.14.6.3}}
+\newlabel{man/cp:options}{{14.6.3}{43}{Options\relax }{subsection.14.6.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.6.4}Exit Status}{43}{subsection.14.6.4}}
+\newlabel{man/cp:exit-status}{{14.6.4}{43}{Exit Status\relax }{subsection.14.6.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.6.5}See Also}{43}{subsection.14.6.5}}
+\newlabel{man/cp:see-also}{{14.6.5}{43}{See Also\relax }{subsection.14.6.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.7}The \textbf {s3qlrm} command}{44}{section.14.7}}
+\newlabel{man/rm::doc}{{14.7}{44}{The \textbf {s3qlrm} command\relax }{section.14.7}{}}
+\newlabel{man/rm:the-s3qlrm-command}{{14.7}{44}{The \textbf {s3qlrm} command\relax }{section.14.7}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.7.1}Synopsis}{44}{subsection.14.7.1}}
+\newlabel{man/rm:synopsis}{{14.7.1}{44}{Synopsis\relax }{subsection.14.7.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.7.2}Description}{44}{subsection.14.7.2}}
+\newlabel{man/rm:description}{{14.7.2}{44}{Description\relax }{subsection.14.7.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.7.3}Options}{44}{subsection.14.7.3}}
+\newlabel{man/rm:options}{{14.7.3}{44}{Options\relax }{subsection.14.7.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.7.4}Exit Status}{44}{subsection.14.7.4}}
+\newlabel{man/rm:exit-status}{{14.7.4}{44}{Exit Status\relax }{subsection.14.7.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.7.5}See Also}{44}{subsection.14.7.5}}
+\newlabel{man/rm:see-also}{{14.7.5}{44}{See Also\relax }{subsection.14.7.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.8}The \textbf {s3qllock} command}{44}{section.14.8}}
+\newlabel{man/lock:the-s3qllock-command}{{14.8}{44}{The \textbf {s3qllock} command\relax }{section.14.8}{}}
+\newlabel{man/lock::doc}{{14.8}{44}{The \textbf {s3qllock} command\relax }{section.14.8}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.8.1}Synopsis}{44}{subsection.14.8.1}}
+\newlabel{man/lock:synopsis}{{14.8.1}{44}{Synopsis\relax }{subsection.14.8.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.8.2}Description}{45}{subsection.14.8.2}}
+\newlabel{man/lock:description}{{14.8.2}{45}{Description\relax }{subsection.14.8.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.8.3}Rationale}{45}{subsection.14.8.3}}
+\newlabel{man/lock:rationale}{{14.8.3}{45}{Rationale\relax }{subsection.14.8.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.8.4}Options}{45}{subsection.14.8.4}}
+\newlabel{man/lock:options}{{14.8.4}{45}{Options\relax }{subsection.14.8.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.8.5}Exit Status}{45}{subsection.14.8.5}}
+\newlabel{man/lock:exit-status}{{14.8.5}{45}{Exit Status\relax }{subsection.14.8.5}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.8.6}See Also}{45}{subsection.14.8.6}}
+\newlabel{man/lock:see-also}{{14.8.6}{45}{See Also\relax }{subsection.14.8.6}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.9}The \textbf {umount.s3ql} command}{46}{section.14.9}}
+\newlabel{man/umount::doc}{{14.9}{46}{The \textbf {umount.s3ql} command\relax }{section.14.9}{}}
+\newlabel{man/umount:the-umount-s3ql-command}{{14.9}{46}{The \textbf {umount.s3ql} command\relax }{section.14.9}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.9.1}Synopsis}{46}{subsection.14.9.1}}
+\newlabel{man/umount:synopsis}{{14.9.1}{46}{Synopsis\relax }{subsection.14.9.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.9.2}Description}{46}{subsection.14.9.2}}
+\newlabel{man/umount:description}{{14.9.2}{46}{Description\relax }{subsection.14.9.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.9.3}Options}{46}{subsection.14.9.3}}
+\newlabel{man/umount:options}{{14.9.3}{46}{Options\relax }{subsection.14.9.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.9.4}Exit Status}{46}{subsection.14.9.4}}
+\newlabel{man/umount:exit-status}{{14.9.4}{46}{Exit Status\relax }{subsection.14.9.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.9.5}See Also}{46}{subsection.14.9.5}}
+\newlabel{man/umount:see-also}{{14.9.5}{46}{See Also\relax }{subsection.14.9.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.10}The \textbf {fsck.s3ql} command}{46}{section.14.10}}
+\newlabel{man/fsck::doc}{{14.10}{46}{The \textbf {fsck.s3ql} command\relax }{section.14.10}{}}
+\newlabel{man/fsck:the-fsck-s3ql-command}{{14.10}{46}{The \textbf {fsck.s3ql} command\relax }{section.14.10}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.10.1}Synopsis}{46}{subsection.14.10.1}}
+\newlabel{man/fsck:synopsis}{{14.10.1}{46}{Synopsis\relax }{subsection.14.10.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.10.2}Description}{47}{subsection.14.10.2}}
+\newlabel{man/fsck:description}{{14.10.2}{47}{Description\relax }{subsection.14.10.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.10.3}Options}{47}{subsection.14.10.3}}
+\newlabel{man/fsck:options}{{14.10.3}{47}{Options\relax }{subsection.14.10.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.10.4}Exit Status}{47}{subsection.14.10.4}}
+\newlabel{man/fsck:exit-status}{{14.10.4}{47}{Exit Status\relax }{subsection.14.10.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.10.5}See Also}{47}{subsection.14.10.5}}
+\newlabel{man/fsck:see-also}{{14.10.5}{47}{See Also\relax }{subsection.14.10.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.11}The \textbf {pcp} command}{47}{section.14.11}}
+\newlabel{man/pcp:the-pcp-command}{{14.11}{47}{The \textbf {pcp} command\relax }{section.14.11}{}}
+\newlabel{man/pcp::doc}{{14.11}{47}{The \textbf {pcp} command\relax }{section.14.11}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.11.1}Synopsis}{47}{subsection.14.11.1}}
+\newlabel{man/pcp:synopsis}{{14.11.1}{47}{Synopsis\relax }{subsection.14.11.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.11.2}Description}{48}{subsection.14.11.2}}
+\newlabel{man/pcp:description}{{14.11.2}{48}{Description\relax }{subsection.14.11.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.11.3}Options}{48}{subsection.14.11.3}}
+\newlabel{man/pcp:options}{{14.11.3}{48}{Options\relax }{subsection.14.11.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.11.4}Exit Status}{48}{subsection.14.11.4}}
+\newlabel{man/pcp:exit-status}{{14.11.4}{48}{Exit Status\relax }{subsection.14.11.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.11.5}See Also}{48}{subsection.14.11.5}}
+\newlabel{man/pcp:see-also}{{14.11.5}{48}{See Also\relax }{subsection.14.11.5}{}}
+\@writefile{toc}{\contentsline {section}{\numberline {14.12}The \textbf {expire\_backups} command}{48}{section.14.12}}
+\newlabel{man/expire_backups::doc}{{14.12}{48}{The \textbf {expire\_backups} command\relax }{section.14.12}{}}
+\newlabel{man/expire_backups:the-expire-backups-command}{{14.12}{48}{The \textbf {expire\_backups} command\relax }{section.14.12}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.12.1}Synopsis}{48}{subsection.14.12.1}}
+\newlabel{man/expire_backups:synopsis}{{14.12.1}{48}{Synopsis\relax }{subsection.14.12.1}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.12.2}Description}{48}{subsection.14.12.2}}
+\newlabel{man/expire_backups:description}{{14.12.2}{48}{Description\relax }{subsection.14.12.2}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.12.3}Options}{49}{subsection.14.12.3}}
+\newlabel{man/expire_backups:options}{{14.12.3}{49}{Options\relax }{subsection.14.12.3}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.12.4}Exit Status}{49}{subsection.14.12.4}}
+\newlabel{man/expire_backups:exit-status}{{14.12.4}{49}{Exit Status\relax }{subsection.14.12.4}{}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {14.12.5}See Also}{49}{subsection.14.12.5}}
+\newlabel{man/expire_backups:see-also}{{14.12.5}{49}{See Also\relax }{subsection.14.12.5}{}}
+\@writefile{toc}{\contentsline {chapter}{\numberline {15}Further Resources / Getting Help}{51}{chapter.15}}
+\@writefile{lof}{\addvspace {10\p@ }}
+\@writefile{lot}{\addvspace {10\p@ }}
+\newlabel{resources::doc}{{15}{51}{Further Resources / Getting Help\relax }{chapter.15}{}}
+\newlabel{resources:further-resources-getting-help}{{15}{51}{Further Resources / Getting Help\relax }{chapter.15}{}}
+\newlabel{resources:resources}{{15}{51}{Further Resources / Getting Help\relax }{chapter.15}{}}
diff --git a/doc/latex/manual.log b/doc/latex/manual.log
index 20029d7..d97b674 100644
--- a/doc/latex/manual.log
+++ b/doc/latex/manual.log
@@ -1,4 +1,4 @@
-This is pdfTeX, Version 3.1415926-1.40.10 (TeX Live 2009/Debian) (format=pdflatex 2011.4.17) 20 MAY 2011 12:20
+This is pdfTeX, Version 3.1415926-1.40.10 (TeX Live 2009/Debian) (format=pdflatex 2011.9.6) 28 SEP 2011 11:35
entering extended mode
%&-line parsing enabled.
**manual.tex
@@ -793,17 +793,13 @@ p/pdftex.map): fontmap entry for `ugqbo8r' already exists, duplicates ignored
] (./manual.toc
LaTeX Font Info: Font shape `T1/ptm/bx/n' in size <10> not available
(Font) Font shape `T1/ptm/b/n' tried instead on input line 2.
-LaTeX Font Info: Try loading font information for T1+pcr on input line 10.
-(/usr/share/texmf-texlive/tex/latex/psnfss/t1pcr.fd
-File: t1pcr.fd 2001/06/04 font definitions for T1/pcr.
-)
LaTeX Font Info: Font shape `T1/phv/bx/n' in size <10> not available
-(Font) Font shape `T1/phv/b/n' tried instead on input line 38.
+(Font) Font shape `T1/phv/b/n' tried instead on input line 40.
pdfTeX warning (ext4): destination with the same identifier (name{page.i}) has
been already used, duplicate ignored
<to be read again>
\relax
-l.38 ...ine {9}Checking for Errors}{23}{chapter.9}
+l.40 ...{\numberline {9}Unmounting}{25}{chapter.9}
[1
])
@@ -819,7 +815,7 @@ l.112 \tableofcontents
Chapter 1.
LaTeX Font Info: Font shape `T1/phv/bx/n' in size <14.4> not available
(Font) Font shape `T1/phv/b/n' tried instead on input line 117.
-LaTeX Font Info: Try loading font information for TS1+ptm on input line 140.
+LaTeX Font Info: Try loading font information for TS1+ptm on input line 139.
(/usr/share/texmf-texlive/tex/latex/psnfss/ts1ptm.fd
File: ts1ptm.fd 2001/06/04 font definitions for TS1/ptm.
@@ -828,107 +824,158 @@ File: ts1ptm.fd 2001/06/04 font definitions for TS1/ptm.
] [2]
Chapter 2.
-[3
+LaTeX Font Info: Try loading font information for T1+pcr on input line 241.
+(/usr/share/texmf-texlive/tex/latex/psnfss/t1pcr.fd
+File: t1pcr.fd 2001/06/04 font definitions for T1/pcr.
+) [3
] [4]
Chapter 3.
+LaTeX Font Info: Font shape `T1/pcr/m/it' in size <9> not available
+(Font) Font shape `T1/pcr/m/sl' tried instead on input line 368.
+[5
-Underfull \hbox (badness 10000) in paragraph at lines 346--347
+]
+Underfull \hbox (badness 10000) in paragraph at lines 413--414
[]
-[5
+[6] [7] [8
]
-LaTeX Font Info: Font shape `T1/pcr/bx/n' in size <14.4> not available
-(Font) Font shape `T1/pcr/b/n' tried instead on input line 402.
-LaTeX Font Info: Font shape `T1/phv/bx/n' in size <12> not available
-(Font) Font shape `T1/phv/b/n' tried instead on input line 462.
-[6] [7]
-LaTeX Font Info: Font shape `T1/pcr/m/it' in size <9> not available
-(Font) Font shape `T1/pcr/m/sl' tried instead on input line 607.
-[8]
Chapter 4.
-[9
-
-] [10
+[9]
+LaTeX Font Info: Font shape `T1/phv/bx/n' in size <12> not available
+(Font) Font shape `T1/phv/b/n' tried instead on input line 548.
+[10] [11] [12
]
Chapter 5.
-[11]
-Underfull \hbox (badness 10000) in paragraph at lines 777--778
+Underfull \hbox (badness 10000) in paragraph at lines 684--686
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
[]
-[12]
+[13] [14
+
+]
Chapter 6.
-[13
-] [14] [15] [16]
+Underfull \hbox (badness 10000) in paragraph at lines 742--744
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
+ []
+
+[15]
+Underfull \hbox (badness 10000) in paragraph at lines 817--818
+
+ []
+
+[16]
Chapter 7.
-[17
-] [18] [19] [20
+Underfull \hbox (badness 10000) in paragraph at lines 847--849
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
+ []
-]
+[17
+
+] [18] [19] [20]
Chapter 8.
-[21] [22
+[21
+
+] [22] [23] [24
]
Chapter 9.
-[23] [24
+[25] [26
]
Chapter 10.
-[25]
-Underfull \hbox (badness 10000) in paragraph at lines 1439--1444
+
+Underfull \hbox (badness 10000) in paragraph at lines 1312--1314
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
+ []
+
+[27] [28
+
+]
+Chapter 11.
+[29]
+Underfull \hbox (badness 10000) in paragraph at lines 1451--1456
[]\T1/ptm/b/n/10 expire_backups \T1/ptm/m/n/10 us-age is sim-ple. It re-quires
back-ups to have names of the forms
[]
-[26] [27] [28
+[30] [31] [32
]
-Chapter 11.
-[29] [30]
Chapter 12.
-[31
-
-] [32]
+[33] [34]
Chapter 13.
-[33
+[35
+
+] [36]
+Chapter 14.
-] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45]
-Underfull \hbox (badness 10000) in paragraph at lines 2672--2677
+Underfull \hbox (badness 10000) in paragraph at lines 1738--1740
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
+ []
+
+[37
+
+]
+Underfull \hbox (badness 10000) in paragraph at lines 1815--1817
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
+ []
+
+[38]
+Underfull \hbox (badness 10000) in paragraph at lines 1894--1896
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
+ []
+
+[39] [40] [41] [42] [43] [44] [45] [46]
+Underfull \hbox (badness 10000) in paragraph at lines 2454--2456
+[]\T1/ptm/m/n/10 Read au-then-ti-ca-tion cre-den-tials from this file (de-fault
+:
+ []
+
+[47] [48]
+Underfull \hbox (badness 10000) in paragraph at lines 2597--2602
[]\T1/ptm/b/n/10 expire_backups \T1/ptm/m/n/10 us-age is sim-ple. It re-quires
back-ups to have names of the forms
[]
-[46] [47] [48
+[49] [50
]
-Chapter 14.
+Chapter 15.
No file manual.ind.
-[49] (./manual.aux) )
+[51] (./manual.aux) )
Here is how much of TeX's memory you used:
- 8345 strings out of 495021
- 113597 string characters out of 1181035
- 200143 words of memory out of 3000000
- 11122 multiletter control sequences out of 15000+50000
- 59136 words of font info for 67 fonts, out of 3000000 for 9000
+ 8325 strings out of 495021
+ 113261 string characters out of 1181036
+ 199733 words of memory out of 3000000
+ 11114 multiletter control sequences out of 15000+50000
+ 58515 words of font info for 65 fonts, out of 3000000 for 9000
29 hyphenation exceptions out of 8191
- 45i,12n,48p,275b,492s stack positions out of 5000i,500n,10000p,200000b,50000s
+ 45i,12n,48p,278b,492s stack positions out of 5000i,500n,10000p,200000b,50000s
{/usr/share/texmf-texlive/fonts/enc/dvips/base/8r.enc}</us
-r/share/texmf-texlive/fonts/type1/urw/courier/ucrb8a.pfb></usr/share/texmf-texl
-ive/fonts/type1/urw/courier/ucrr8a.pfb></usr/share/texmf-texlive/fonts/type1/ur
-w/courier/ucrro8a.pfb></usr/share/texmf-texlive/fonts/type1/urw/helvetic/uhvb8a
-.pfb></usr/share/texmf-texlive/fonts/type1/urw/helvetic/uhvbo8a.pfb></usr/share
-/texmf-texlive/fonts/type1/urw/times/utmb8a.pfb></usr/share/texmf-texlive/fonts
-/type1/urw/times/utmr8a.pfb></usr/share/texmf-texlive/fonts/type1/urw/times/utm
-ri8a.pfb>
-Output written on manual.pdf (53 pages, 289155 bytes).
+r/share/texmf-texlive/fonts/type1/urw/courier/ucrr8a.pfb></usr/share/texmf-texl
+ive/fonts/type1/urw/courier/ucrro8a.pfb></usr/share/texmf-texlive/fonts/type1/u
+rw/helvetic/uhvb8a.pfb></usr/share/texmf-texlive/fonts/type1/urw/helvetic/uhvbo
+8a.pfb></usr/share/texmf-texlive/fonts/type1/urw/times/utmb8a.pfb></usr/share/t
+exmf-texlive/fonts/type1/urw/times/utmr8a.pfb></usr/share/texmf-texlive/fonts/t
+ype1/urw/times/utmri8a.pfb>
+Output written on manual.pdf (55 pages, 286628 bytes).
PDF statistics:
- 781 PDF objects out of 1000 (max. 8388607)
- 224 named destinations out of 1000 (max. 500000)
- 465 words of extra memory for PDF output out of 10000 (max. 10000000)
+ 804 PDF objects out of 1000 (max. 8388607)
+ 214 named destinations out of 1000 (max. 500000)
+ 489 words of extra memory for PDF output out of 10000 (max. 10000000)
diff --git a/doc/latex/manual.out b/doc/latex/manual.out
index 16ea0d1..e1db91b 100644
--- a/doc/latex/manual.out
+++ b/doc/latex/manual.out
@@ -4,55 +4,58 @@
\BOOKMARK [0][-]{chapter.2}{Installation}{}
\BOOKMARK [1][-]{section.2.1}{Dependencies}{chapter.2}
\BOOKMARK [1][-]{section.2.2}{Installing S3QL}{chapter.2}
-\BOOKMARK [0][-]{chapter.3}{Storage Backends}{}
-\BOOKMARK [1][-]{section.3.1}{On Backend Reliability}{chapter.3}
-\BOOKMARK [1][-]{section.3.2}{The authinfo file}{chapter.3}
-\BOOKMARK [1][-]{section.3.3}{Consistency Guarantees}{chapter.3}
-\BOOKMARK [1][-]{section.3.4}{The Amazon S3 Backend}{chapter.3}
-\BOOKMARK [1][-]{section.3.5}{The Local Backend}{chapter.3}
-\BOOKMARK [1][-]{section.3.6}{The SFTP Backend}{chapter.3}
-\BOOKMARK [0][-]{chapter.4}{File System Creation}{}
-\BOOKMARK [0][-]{chapter.5}{Managing Buckets}{}
-\BOOKMARK [1][-]{section.5.1}{Changing the Passphrase}{chapter.5}
-\BOOKMARK [1][-]{section.5.2}{Upgrading the file system}{chapter.5}
-\BOOKMARK [1][-]{section.5.3}{Deleting a file system}{chapter.5}
-\BOOKMARK [1][-]{section.5.4}{Restoring Metadata Backups}{chapter.5}
-\BOOKMARK [0][-]{chapter.6}{Mounting}{}
-\BOOKMARK [1][-]{section.6.1}{Storing Encryption Passwords}{chapter.6}
-\BOOKMARK [1][-]{section.6.2}{Compression Algorithms}{chapter.6}
-\BOOKMARK [1][-]{section.6.3}{Parallel Compression}{chapter.6}
-\BOOKMARK [1][-]{section.6.4}{Notes about Caching}{chapter.6}
-\BOOKMARK [1][-]{section.6.5}{Automatic Mounting}{chapter.6}
-\BOOKMARK [0][-]{chapter.7}{Advanced S3QL Features}{}
-\BOOKMARK [1][-]{section.7.1}{Snapshotting and Copy-on-Write}{chapter.7}
-\BOOKMARK [1][-]{section.7.2}{Getting Statistics}{chapter.7}
-\BOOKMARK [1][-]{section.7.3}{Immutable Trees}{chapter.7}
-\BOOKMARK [1][-]{section.7.4}{Fast Recursive Removal}{chapter.7}
-\BOOKMARK [1][-]{section.7.5}{Runtime Configuration}{chapter.7}
-\BOOKMARK [0][-]{chapter.8}{Unmounting}{}
-\BOOKMARK [0][-]{chapter.9}{Checking for Errors}{}
-\BOOKMARK [0][-]{chapter.10}{Contributed Programs}{}
-\BOOKMARK [1][-]{section.10.1}{benchmark.py}{chapter.10}
-\BOOKMARK [1][-]{section.10.2}{s3\137copy.py}{chapter.10}
-\BOOKMARK [1][-]{section.10.3}{pcp.py}{chapter.10}
-\BOOKMARK [1][-]{section.10.4}{s3\137backup.sh}{chapter.10}
-\BOOKMARK [1][-]{section.10.5}{expire\137backups.py}{chapter.10}
-\BOOKMARK [1][-]{section.10.6}{s3ql.conf}{chapter.10}
-\BOOKMARK [0][-]{chapter.11}{Tips \046 Tricks}{}
-\BOOKMARK [1][-]{section.11.1}{Permanently mounted backup file system}{chapter.11}
-\BOOKMARK [1][-]{section.11.2}{Improving copy performance}{chapter.11}
-\BOOKMARK [0][-]{chapter.12}{Known Issues}{}
-\BOOKMARK [0][-]{chapter.13}{Manpages}{}
-\BOOKMARK [1][-]{section.13.1}{The mkfs.s3ql command}{chapter.13}
-\BOOKMARK [1][-]{section.13.2}{The s3qladm command}{chapter.13}
-\BOOKMARK [1][-]{section.13.3}{The mount.s3ql command}{chapter.13}
-\BOOKMARK [1][-]{section.13.4}{The s3qlstat command}{chapter.13}
-\BOOKMARK [1][-]{section.13.5}{The s3qlctrl command}{chapter.13}
-\BOOKMARK [1][-]{section.13.6}{The s3qlcp command}{chapter.13}
-\BOOKMARK [1][-]{section.13.7}{The s3qlrm command}{chapter.13}
-\BOOKMARK [1][-]{section.13.8}{The s3qllock command}{chapter.13}
-\BOOKMARK [1][-]{section.13.9}{The umount.s3ql command}{chapter.13}
-\BOOKMARK [1][-]{section.13.10}{The fsck.s3ql command}{chapter.13}
-\BOOKMARK [1][-]{section.13.11}{The pcp command}{chapter.13}
-\BOOKMARK [1][-]{section.13.12}{The expire\137backups command}{chapter.13}
-\BOOKMARK [0][-]{chapter.14}{Further Resources / Getting Help}{}
+\BOOKMARK [0][-]{chapter.3}{General Information}{}
+\BOOKMARK [1][-]{section.3.1}{Terminology}{chapter.3}
+\BOOKMARK [1][-]{section.3.2}{Storing Authentication Information}{chapter.3}
+\BOOKMARK [1][-]{section.3.3}{On Backend Reliability}{chapter.3}
+\BOOKMARK [0][-]{chapter.4}{Storage Backends}{}
+\BOOKMARK [1][-]{section.4.1}{Google Storage}{chapter.4}
+\BOOKMARK [1][-]{section.4.2}{Amazon S3}{chapter.4}
+\BOOKMARK [1][-]{section.4.3}{S3 compatible}{chapter.4}
+\BOOKMARK [1][-]{section.4.4}{Local}{chapter.4}
+\BOOKMARK [1][-]{section.4.5}{SSH/SFTP}{chapter.4}
+\BOOKMARK [0][-]{chapter.5}{File System Creation}{}
+\BOOKMARK [0][-]{chapter.6}{Managing Buckets}{}
+\BOOKMARK [1][-]{section.6.1}{Changing the Passphrase}{chapter.6}
+\BOOKMARK [1][-]{section.6.2}{Upgrading the file system}{chapter.6}
+\BOOKMARK [1][-]{section.6.3}{Deleting a file system}{chapter.6}
+\BOOKMARK [1][-]{section.6.4}{Restoring Metadata Backups}{chapter.6}
+\BOOKMARK [0][-]{chapter.7}{Mounting}{}
+\BOOKMARK [1][-]{section.7.1}{Compression Algorithms}{chapter.7}
+\BOOKMARK [1][-]{section.7.2}{Parallel Compression}{chapter.7}
+\BOOKMARK [1][-]{section.7.3}{Notes about Caching}{chapter.7}
+\BOOKMARK [1][-]{section.7.4}{Automatic Mounting}{chapter.7}
+\BOOKMARK [0][-]{chapter.8}{Advanced S3QL Features}{}
+\BOOKMARK [1][-]{section.8.1}{Snapshotting and Copy-on-Write}{chapter.8}
+\BOOKMARK [1][-]{section.8.2}{Getting Statistics}{chapter.8}
+\BOOKMARK [1][-]{section.8.3}{Immutable Trees}{chapter.8}
+\BOOKMARK [1][-]{section.8.4}{Fast Recursive Removal}{chapter.8}
+\BOOKMARK [1][-]{section.8.5}{Runtime Configuration}{chapter.8}
+\BOOKMARK [0][-]{chapter.9}{Unmounting}{}
+\BOOKMARK [0][-]{chapter.10}{Checking for Errors}{}
+\BOOKMARK [0][-]{chapter.11}{Contributed Programs}{}
+\BOOKMARK [1][-]{section.11.1}{benchmark.py}{chapter.11}
+\BOOKMARK [1][-]{section.11.2}{s3\137copy.py}{chapter.11}
+\BOOKMARK [1][-]{section.11.3}{pcp.py}{chapter.11}
+\BOOKMARK [1][-]{section.11.4}{s3\137backup.sh}{chapter.11}
+\BOOKMARK [1][-]{section.11.5}{expire\137backups.py}{chapter.11}
+\BOOKMARK [1][-]{section.11.6}{s3ql.conf}{chapter.11}
+\BOOKMARK [0][-]{chapter.12}{Tips \046 Tricks}{}
+\BOOKMARK [1][-]{section.12.1}{SSH Backend}{chapter.12}
+\BOOKMARK [1][-]{section.12.2}{Permanently mounted backup file system}{chapter.12}
+\BOOKMARK [1][-]{section.12.3}{Improving copy performance}{chapter.12}
+\BOOKMARK [0][-]{chapter.13}{Known Issues}{}
+\BOOKMARK [0][-]{chapter.14}{Manpages}{}
+\BOOKMARK [1][-]{section.14.1}{The mkfs.s3ql command}{chapter.14}
+\BOOKMARK [1][-]{section.14.2}{The s3qladm command}{chapter.14}
+\BOOKMARK [1][-]{section.14.3}{The mount.s3ql command}{chapter.14}
+\BOOKMARK [1][-]{section.14.4}{The s3qlstat command}{chapter.14}
+\BOOKMARK [1][-]{section.14.5}{The s3qlctrl command}{chapter.14}
+\BOOKMARK [1][-]{section.14.6}{The s3qlcp command}{chapter.14}
+\BOOKMARK [1][-]{section.14.7}{The s3qlrm command}{chapter.14}
+\BOOKMARK [1][-]{section.14.8}{The s3qllock command}{chapter.14}
+\BOOKMARK [1][-]{section.14.9}{The umount.s3ql command}{chapter.14}
+\BOOKMARK [1][-]{section.14.10}{The fsck.s3ql command}{chapter.14}
+\BOOKMARK [1][-]{section.14.11}{The pcp command}{chapter.14}
+\BOOKMARK [1][-]{section.14.12}{The expire\137backups command}{chapter.14}
+\BOOKMARK [0][-]{chapter.15}{Further Resources / Getting Help}{}
diff --git a/doc/latex/manual.tex b/doc/latex/manual.tex
index fe2532b..250ef72 100644
--- a/doc/latex/manual.tex
+++ b/doc/latex/manual.tex
@@ -12,8 +12,8 @@
\title{S3QL Documentation}
-\date{May 20, 2011}
-\release{1.0.1}
+\date{September 28, 2011}
+\release{1.2}
\author{Nikolaus Rath}
\newcommand{\sphinxlogo}{}
\renewcommand{\releasename}{Release}
@@ -116,13 +116,12 @@
\chapter{About S3QL}
\label{about:about-s3ql}\label{about::doc}\label{about:s3ql-user-s-guide}
-S3QL is a file system that stores all its data online. It supports
-\href{http://aws.amazon.com/s3AmazonS3}{Amazon S3} as well as arbitrary
-SFTP servers and effectively provides you with a hard disk of dynamic,
-infinite capacity that can be accessed from any computer with internet
-access.
+S3QL is a file system that stores all its data online using storage
+services like \href{http://code.google.com/apis/storage/}{Google Storage}, \href{http://aws.amazon.com/s3AmazonS3}{Amazon S3} or \href{http://openstack.org/projects/storage/}{OpenStack}. S3QL effectively provides
+a hard disk of dynamic, infinite capacity that can be accessed from
+any computer with internet access.
-S3QL is providing a standard, full featured UNIX file system that is
+S3QL is a standard conforming, full featured UNIX file system that is
conceptually indistinguishable from any local file system.
Furthermore, S3QL has additional features like compression,
encryption, data de-duplication, immutable trees and snapshotting
@@ -232,14 +231,10 @@ already provides a suitable packages and only install from source if
that is not the case.
\begin{itemize}
\item {}
-Kernel version 2.6.9 or newer. Starting with kernel 2.6.26
-you will get significantly better write performance, so you should
-actually use \emph{2.6.26 or newer whenever possible}.
-
-\item {}
-The \href{http://fuse.sourceforge.net/}{FUSE Library} should already be
-installed on your system. However, you have to make sure that you
-have at least version 2.8.0.
+Kernel: Linux 2.6.9 or newer or FreeBSD with \href{http://www.freshports.org/sysutils/fusefs-kmod/}{FUSE4BSD}. Starting with
+kernel 2.6.26 you will get significantly better write performance,
+so under Linux you should actually use \emph{2.6.26 or newer whenever
+possible}.
\item {}
The \href{http://pypi.python.org/pypi/pycryptopp}{PyCrypto++ Python Module}. To check if this module
@@ -278,12 +273,6 @@ Note that earlier S3QL versions shipped with a builtin version of
this module. If you are upgrading from such a version, make sure to
completely remove the old S3QL version first.
-\item {}
-If you want to use the SFTP backend, then you also need the
-\href{http://www.lag.net/paramiko/}{Paramiko Python Module}. To check
-if this module is installed, try to execute \code{python -c 'import
-paramiko'}.
-
\end{itemize}
@@ -321,17 +310,95 @@ setup.py install -{-}user}. In this case you should make sure that
\end{itemize}
-\chapter{Storage Backends}
-\label{backends::doc}\label{backends:storage-backends}
-S3QL can use different protocols to store the file system data.
-Independent of the backend that you use, the place where your file
-system data is being stored is called a \emph{bucket}. (This is mostly for
-historical reasons, since initially S3QL supported only the Amazon S3
-backend).
+\chapter{General Information}
+\label{general:general-information}\label{general::doc}
+
+\section{Terminology}
+\label{general:terminology}
+S3QL can store data at different service providers and using different
+protocols. The term \emph{backend} refers to both the part of S3QL that
+implements communication with a specific storage service and the
+storage service itself. Most backends can hold more than one S3QL file
+system and thus require some additional information that specifies the
+file system location within the backend. This location is called a
+\emph{bucket} (for historical reasons).
+
+Many S3QL commands expect a \emph{storage url} as a parameter. A storage
+url specifies both the backend and the bucket and thus uniquely
+identifies an S3QL file system. The form of the storage url depends on
+the backend and is described together with the
+{\hyperref[backends:storage-backends]{\emph{Storage Backends}}}.
+
+
+\section{Storing Authentication Information}
+\label{general:storing-authentication-information}\label{general:bucket-pw}
+Normally, S3QL reads username and password for the backend as well as
+an encryption passphrase for the bucket from the terminal. Most
+commands also accept an \code{-{-}authfile} parameter that can be
+used to read this information from a file instead.
+
+The authentication file consists of sections, led by a \code{{[}section{]}}
+header and followed by \code{name: value} entries. The section headers
+themselves are not used by S3QL but have to be unique within the file.
+
+In each section, the following entries can be defined:
+\begin{quote}\begin{description}
+\item[{storage-url}] \leavevmode
+Specifies the storage url to which this section applies. If a
+storage url starts with the value of this entry, the section is
+considered applicable.
+
+\item[{backend-login}] \leavevmode
+Specifies the username to use for authentication with the backend.
+
+\item[{backend-password}] \leavevmode
+Specifies the password to use for authentication with the backend.
+
+\item[{bucket-passphrase}] \leavevmode
+Specifies the passphrase to use to decrypt the bucket (if it is
+encrypted).
+
+\end{description}\end{quote}
+
+When reading the authentication file, S3QL considers every applicable
+section in order and uses the last value that it found for each entry.
+For example, consider the following authentication file:
+
+\begin{Verbatim}[commandchars=\\\{\}]
+\PYG{g+ge}{[s3]}
+\PYG{l}{storage-url: s3://}
+\PYG{l}{backend-login: joe}
+\PYG{l}{backend-password: notquitesecret}
+
+\PYG{g+ge}{[bucket1]}
+\PYG{l}{storage-url: s3://joes-first-bucket}
+\PYG{l}{bucket-passphrase: neitheristhis}
+
+\PYG{g+ge}{[bucket2]}
+\PYG{l}{storage-url: s3://joes-second-bucket}
+\PYG{l}{bucket-passphrase: swordfish}
+
+\PYG{g+ge}{[bucket3]}
+\PYG{l}{storage-url: s3://joes-second-bucket/with-prefix}
+\PYG{l}{backend-login: bill}
+\PYG{l}{backend-password: bi23ll}
+\PYG{l}{bucket-passphrase: ll23bi}
+\end{Verbatim}
+
+With this authentication file, S3QL would try to log in as ``joe''
+whenever the s3 backend is used, except when accessing a storage url
+that begins with ``s3://joes-second-bucket/with-prefix''. In that case,
+the last section becomes active and S3QL would use the ``bill''
+credentials. Furthermore, bucket encryption passphrases will be used
+for storage urls that start with ``s3://joes-first-bucket'' or
+``s3://joes-second-bucket''.
+
+The authentication file is parsed by the \href{http://docs.python.org/library/configparser.html}{Python ConfigParser
+module}.
\section{On Backend Reliability}
-\label{backends:on-backend-reliability}
+\label{general:backend-reliability}\label{general:on-backend-reliability}
S3QL has been designed for use with a storage backend where data loss
is so infrequent that it can be completely neglected (e.g. the Amazon
S3 backend). If you decide to use a less reliable backend, you should
@@ -394,229 +461,209 @@ a ``checkpoint'': data loss in the backend that occurred before running
\code{fsck.s3ql} can not affect any file system operations performed after
running \code{fsck.s3ql}.
-Nevertheless, (as said at the beginning) the recommended way to use
-S3QL is in combination with a sufficiently reliable storage backend.
-In that case none of the above will ever be a concern.
+Nevertheless, the recommended way to use S3QL is in combination with a
+sufficiently reliable storage backend. In that case none of the above
+will ever be a concern.
+
+
+\chapter{Storage Backends}
+\label{backends:id1}\label{backends::doc}\label{backends:storage-backends}
+The following backends are currently available in S3QL:
+
+\section{Google Storage}
+\label{backends:google-storage}
+\href{http://code.google.com/apis/storage/}{Google Storage} is an online
+storage service offered by Google. It is the most feature-rich service
+supported by S3QL and S3QL offers the best performance when used with
+the Google Storage backend.
-\section{The \texttt{authinfo} file}
-\label{backends:the-authinfo-file}
-Most backends first try to read the file \code{\textasciitilde{}/.s3ql/authinfo} to determine
-the username and password for connecting to the remote host. If this
-fails, both username and password are read from the terminal.
+To use the Google Storage backend, you need to have (or sign up for) a
+Google account, and then \href{http://code.google.com/apis/storage/docs/signup.html}{activate Google Storage} for your
+account. The account is free, you will pay only for the amount of
+storage and traffic that you actually use. Once you have created the
+account, make sure to \href{http://code.google.com/apis/storage/docs/reference/v1/apiversion1.html\#enabling}{activate legacy access}.
-The \code{authinfo} file has to contain entries of the form
+To create a Google Storage bucket, you can use e.g. the \href{https://sandbox.google.com/storage/}{Google
+Storage Manager}. The
+storage URL for accessing the bucket in S3QL is then
\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{backend }\PYG{n+nv}{\textless{}backend\textgreater{}}\PYG{l}{ machine }\PYG{n+nv}{\textless{}host\textgreater{}}\PYG{l}{ login }\PYG{n+nv}{\textless{}user\textgreater{}}\PYG{l}{ password }\PYG{n+nv}{\textless{}password\textgreater{}}
+\PYG{l}{gs://}\PYG{n+nv}{\textless{}bucketname\textgreater{}}\PYG{l}{/}\PYG{n+nv}{\textless{}prefix\textgreater{}}
\end{Verbatim}
-So to use the login \code{joe} with password \code{jibbadup} when using the FTP
-backend to connect to the host \code{backups.joesdomain.com}, you would
-specify
+Here \emph{bucketname} is the name of the bucket, and \emph{prefix} can be
+an arbitrary prefix that will be prepended to all object names used by
+S3QL. This allows you to store several S3QL file systems in the same
+Google Storage bucket.
+
+Note that the backend login and password for accessing your Google
+Storage bucket are not your Google account name and password, but the
+\emph{Google Storage developer access key} and \emph{Google Storage developer
+secret} that you can manage with the \href{https://code.google.com/apis/console/\#:storage:legacy}{Google Storage key management
+tool}.
+
+If you would like S3QL to connect using HTTPS instead of standard
+HTTP, start the storage url with \code{gss://} instead of \code{gs://}. Note
+that at this point S3QL does not perform any server certificate
+validation (see \href{http://code.google.com/p/s3ql/issues/detail?id=267}{issue 267}).
+
+
+\section{Amazon S3}
+\label{backends:amazon-s3}
+\href{http://aws.amazon.com/s3}{Amazon S3} is the online storage service
+offered by \href{http://aws.amazon.com/}{Amazon Web Services (AWS)}. To
+use the S3 backend, you first need to sign up for an AWS account. The
+account is free, you will pay only for the amount of storage and
+traffic that you actually use. After that, you need to create a bucket
+that will hold the S3QL file system, e.g. using the \href{https://console.aws.amazon.com/s3/home}{AWS Management
+Console}. For best
+performance, it is recommend to create the bucket in the
+geographically closest storage region, but not the US Standard
+region (see below).
+
+The storage URL for accessing S3 buckets in S3QL has the form
\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{backend ftp machine backups.joesdomain.com login joe password jibbadup}
+\PYG{l}{s3://}\PYG{n+nv}{\textless{}bucketname\textgreater{}}\PYG{l}{/}\PYG{n+nv}{\textless{}prefix\textgreater{}}
\end{Verbatim}
+Here \emph{bucketname} is the name of the bucket, and \emph{prefix} can be
+an arbitrary prefix that will be prepended to all object names used by
+S3QL. This allows you to store several S3QL file systems in the same
+S3 bucket.
+
+Note that the backend login and password for accessing S3 are not the
+user id and password that you use to log into the Amazon Webpage, but
+the \emph{AWS access key id} and \emph{AWS secret access key} shown under \href{https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8\&action=access-key}{My
+Account/Access Identifiers}.
+
+If you would like S3QL to connect using HTTPS instead of standard
+HTTP, start the storage url with \code{s3s://} instead of \code{s3://}. Note
+that, as of May 2011, Amazon S3 is faster when accessed using a
+standard HTTP connection, and that S3QL does not perform any server
+certificate validation (see \href{http://code.google.com/p/s3ql/issues/detail?id=267}{issue 267}).
-\section{Consistency Guarantees}
-\label{backends:consistency-guarantees}
-The different backends provide different types of \emph{consistency
-guarantees}. Informally, a consistency guarantee tells you how fast
-the backend will apply changes to the stored data.
-S3QL defines the following three levels:
+\subsection{Reduced Redundancy Storage (RRS)}
+\label{backends:reduced-redundancy-storage-rrs}
+S3QL does not allow the use of \href{http://aws.amazon.com/s3/\#protecting}{reduced redundancy storage}. The reason for that is a
+combination of three factors:
\begin{itemize}
\item {}
-\textbf{Read-after-Write Consistency.} This is the strongest consistency
-guarantee. If a backend offers read-after-write consistency, it
-guarantees that as soon as you have committed any changes to the
-backend, subsequent requests will take into account these changes.
+RRS has a relatively low reliability, on average you loose one
+out of every ten-thousand objects a year. So you can expect to
+occasionally loose some data.
\item {}
-\textbf{Read-after-Create Consistency.} If a backend provides only
-read-after-create consistency, only the creation of a new object is
-guaranteed to be taken into account for subsequent requests. This
-means that, for example, if you overwrite data in an existing
-object, subsequent requests may still return the old data for a
-certain period of time.
+When \code{fsck.s3ql} asks S3 for a list of the stored objects, this list
+includes even those objects that have been lost. Therefore
+\code{fsck.s3ql} \emph{can not detect lost objects} and lost data will only
+become apparent when you try to actually read from a file whose data
+has been lost. This is a (very unfortunate) peculiarity of Amazon
+S3.
\item {}
-\textbf{Eventual consistency.} This is the lowest consistency level.
-Basically, any changes that you make to the backend may not be
-visible for a certain amount of time after the change has been made.
-However, you are guaranteed that no change will be lost. All changes
-will \emph{eventually} become visible.
-
-.
+Due to the data de-duplication feature of S3QL, unnoticed lost
+objects may cause subsequent data loss later in time (see
+{\hyperref[general:backend-reliability]{\emph{On Backend Reliability}}} for details).
\end{itemize}
-As long as your backend provides read-after-write or read-after-create
-consistency, you do not have to worry about consistency guarantees at
-all. However, if you plan to use a backend with only eventual
-consistency, you have to be a bit careful in some situations.
-
-
-\subsection{Dealing with Eventual Consistency}
-\label{backends:dealing-with-eventual-consistency}\label{backends:eventual-consistency}
-\begin{notice}{note}{Note:}
-The following applies only to storage backends that do not provide
-read-after-create or read-after-write consistency. Currently,
-this is only the Amazon S3 backend \emph{if used with the US-Standard
-storage region}. If you use a different storage backend, or the S3
-backend with a different storage region, this section does not apply
-to you.
-\end{notice}
-While the file system is mounted, S3QL is able to automatically handle
-all issues related to the weak eventual consistency guarantee.
-However, some issues may arise during the mount process and when the
-file system is checked.
+\subsection{Potential issues when using the US Standard storage region}
+\label{backends:potential-issues-when-using-the-us-standard-storage-region}
+In the US Standard storage region, Amazon S3 does not guarantee read
+after create consistency. This means that after a new object has been
+stored, requests to read this object may still fail for a little
+while. While the file system is mounted, S3QL is able to automatically
+handle all issues related to this so-called eventual consistency.
+However, problems may arise during the mount process and when the file
+system is checked:
Suppose that you mount the file system, store some new data, delete
-some old data and unmount it again. Now remember that eventual
-consistency means that there is no guarantee that these changes will
-be visible immediately. At least in theory it is therefore possible
-that if you mount the file system again, S3QL does not see any of the
-changes that you have done and presents you an ``old version'' of the
-file system without them. Even worse, if you notice the problem and
-unmount the file system, S3QL will upload the old status (which S3QL
-necessarily has to consider as current) and thereby permanently
-override the newer version (even though this change may not become
-immediately visible either).
-
-The same problem applies when checking the file system. If the backend
+some old data and unmount it again. Now there is no guarantee that
+these changes will be visible immediately. At least in theory it is
+therefore possible that if you mount the file system again, S3QL
+does not see any of the changes that you have done and presents you
+an ``old version'' of the file system without them. Even worse, if you
+notice the problem and unmount the file system, S3QL will upload the
+old status (which S3QL necessarily has to consider as current) and
+thereby permanently override the newer version (even though this
+change may not become immediately visible either).
+
+The same problem applies when checking the file system. If S3
provides S3QL with only partially updated data, S3QL has no way to
find out if this a real consistency problem that needs to be fixed or
if it is only a temporary problem that will resolve itself
automatically (because there are still changes that have not become
visible yet).
-While this may seem to be a rather big problem, the likelihood of it
-to occur is rather low. In practice, most storage providers rarely
-need more than a few seconds to apply incoming changes, so to trigger
-this problem one would have to unmount and remount the file system in
-a very short time window. Many people therefore make sure that they
-wait a few minutes between successive mounts (or file system checks)
-and decide that the remaining risk is negligible.
-
-Nevertheless, the eventual consistency guarantee does not impose an
-upper limit on the time that it may take for change to become visible.
-Therefore there is no ``totally safe'' waiting time that would totally
-eliminate this problem; a theoretical possibility always remains.
-
-
-\section{The Amazon S3 Backend}
-\label{backends:the-amazon-s3-backend}
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \code{s3://\textless{}bucketname\textgreater{}}. Bucket names must conform to the \href{http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/BucketRestrictions.html}{S3
-Bucket Name Restrictions}.
-
-The S3 backend offers exceptionally strong reliability guarantees. As
-of August 2010, Amazon guarantees a durability of 99.999999999\% per
-year. In other words, if you store a thousand million objects then on
-average you would loose less than one object in a hundred years.
-
-The Amazon S3 backend provides read-after-create consistency for the
-EU, Asia-Pacific and US-West storage regions. \emph{For the US-Standard
-storage region, Amazon S3 provides only eventual consistency} (please
-refer to {\hyperref[backends:eventual-consistency]{\emph{Dealing with Eventual Consistency}}} for information about
-what this entails).
-
-When connecting to Amazon S3, S3QL uses an unencrypted HTTP
-connection, so if you want your data to stay confidential, you have
-to create the S3QL file system with encryption (this is also the default).
-
-When reading the authentication information for the S3 backend from
-the \code{authinfo} file, the \code{host} field is ignored, i.e. the first entry
-with \code{s3} as a backend will be used. For example
-
-\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{backend s3 machine any login myAWSaccessKeyId password myAwsSecretAccessKey}
-\end{Verbatim}
-
-Note that the bucket names come from a global pool, so chances are
-that your favorite name has already been taken by another S3 user.
-Usually a longer bucket name containing some random numbers, like
-\code{19283712\_yourname\_s3ql}, will work better.
-
-If you do not already have one, you need to obtain an Amazon S3
-account from \href{http://aws.amazon.com/}{Amazon AWS}. The account is
-free, you will pay only for the amount of storage that you actually
-use.
-
-Note that the login and password for accessing S3 are not the user id
-and password that you use to log into the Amazon Webpage, but the ``AWS
-access key id'' and ``AWS secret access key'' shown under \href{https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8\&action=access-key}{My
-Account/Access Identifiers}.
-
-\begin{notice}{note}{Note:}
-S3QL also allows you to use \href{http://aws.amazon.com/s3/\#protecting}{reduced redundancy storage} by using \code{s3rr://}
-instead of \code{s3://} in the storage url. However, this not
-recommended. The reason is a combination of three factors:
-\begin{itemize}
-\item {}
-RRS has a relatively low reliability, on average you loose one
-out of every ten-thousand objects a year. So you can expect to
-occasionally loose some data.
-
-\item {}
-When \code{fsck.s3ql} asks Amazon S3 for a list of the stored objects,
-this list includes even those objects that have been lost.
-Therefore \code{fsck.s3ql} \emph{can not detect lost objects} and lost data
-will only become apparent when you try to actually read from a
-file whose data has been lost. This is a (very unfortunate)
-peculiarity of Amazon S3.
+The likelihood of this to happen is rather low. In practice, most
+objects are ready for retrieval just a few seconds after they have
+been stored, so to trigger this problem one would have to unmount and
+remount the file system in a very short time window. However, since S3
+does not place any upper limit on the length of this window, it is
+recommended to not place S3QL buckets in the US Standard storage
+region. As of May 2011, all other storage regions provide stronger
+consistency guarantees that completely eliminate any of the described
+problems.
-\item {}
-Due to the data de-duplication feature of S3QL, unnoticed lost
-objects may cause subsequent data loss later in time (see {\hyperref[backends:on-backend-reliability]{On
-Backend Reliability}} for details).
-\end{itemize}
+\section{S3 compatible}
+\label{backends:s3-compatible}
+S3QL is also able to access other, S3 compatible storage services for
+which no specific backend exists. Note that when accessing such
+services, only the lowest common denominator of available features can
+be used, so it is generally recommended to use a service specific
+backend instead.
-In other words, you should really only store an S3QL file system
-using RRS if you know exactly what you are getting into.
-\end{notice}
+The storage URL for accessing an arbitrary S3 compatible storage
+service is
+\begin{Verbatim}[commandchars=\\\{\}]
+\PYG{l}{s3c://}\PYG{n+nv}{\textless{}hostname\textgreater{}}\PYG{l}{:}\PYG{n+nv}{\textless{}port\textgreater{}}\PYG{l}{/}\PYG{n+nv}{\textless{}bucketname\textgreater{}}\PYG{l}{/}\PYG{n+nv}{\textless{}prefix\textgreater{}}
+\end{Verbatim}
-\section{The Local Backend}
-\label{backends:the-local-backend}
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\code{local://\textless{}path\textgreater{}}. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \code{local:///var/archive}.
+or
-The local backend provides read-after-write consistency.
+\begin{Verbatim}[commandchars=\\\{\}]
+\PYG{l}{s3cs://}\PYG{n+nv}{\textless{}hostname\textgreater{}}\PYG{l}{:}\PYG{n+nv}{\textless{}port\textgreater{}}\PYG{l}{/}\PYG{n+nv}{\textless{}bucketname\textgreater{}}\PYG{l}{/}\PYG{n+nv}{\textless{}prefix\textgreater{}}
+\end{Verbatim}
+to use HTTPS connections. Note, however, that at this point S3QL does
+not verify the server certificate (cf. \href{http://code.google.com/p/s3ql/issues/detail?id=267}{issue 267}).
-\section{The SFTP Backend}
-\label{backends:the-sftp-backend}
-The SFTP backend uses the SFTP protocol, which is a file transfer
-protocol similar to ftp, but uses an encrypted SSH connection.
-It provides read-after-write consistency.
-Note that the SFTP backend is rather slow and has not been tested
-as extensively as the S3 and Local backends.
+\section{Local}
+\label{backends:local}
+S3QL is also able to store its data on the local file system. This can
+be used to backup data on external media, or to access external
+services that S3QL can not talk to directly (e.g., it is possible to
+store data over SSH by first mounting the remote system using
+\href{http://fuse.sourceforge.net/sshfs.html}{sshfs}, then using the local backend to store the data in the sshfs
+mountpoint).
-The storage URL for SFTP connections has the form
+The storage URL for local storage is
\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{sftp://}\PYG{n+nv}{\textless{}host\textgreater{}}\PYG{g+ge}{[:port]}\PYG{l}{/}\PYG{n+nv}{\textless{}path\textgreater{}}
+\PYG{l}{local://}\PYG{n+nv}{\textless{}path\textgreater{}}
\end{Verbatim}
-The SFTP backend will always ask you for a password if you haven't
-defined one in \code{\textasciitilde{}/.s3ql/authinfo}. However, public key authentication
-is tried first and the password will only be used if the public key
-authentication fails.
+Note that you have to write three consecutive slashes to specify an
+absolute path, e.g. \code{local:///var/archive}. Also, relative paths will
+automatically be converted to absolute paths before the authentication
+file is read, i.e. if you are in the \code{/home/john} directory and try to
+mount \code{local://bucket}, the corresponding section in the
+authentication file must match the storage url
+\code{local:///home/john/bucket}.
+
-The public and private keys will be read from the standard files in
-\code{\textasciitilde{}/.ssh/}. Note that S3QL will refuse to connect to a computer with
-unknown host key; to add the key to your local keyring you have to
-establish a connection to that computer with the standard SSH command
-line programs first.
+\section{SSH/SFTP}
+\label{backends:ssh-sftp}
+Previous versions of S3QL included an SSH/SFTP backend. With newer
+S3QL versions, it is recommended to instead combine the local backend
+with \href{http://fuse.sourceforge.net/sshfs.html}{sshfs} (cf. {\hyperref[tips:ssh-tipp]{\emph{SSH Backend}}}).
\chapter{File System Creation}
@@ -631,27 +678,19 @@ following syntax:
This command accepts the following options:
\begin{quote}
\begin{optionlist}{3cm}
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication
-info. (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file (default:
+\code{\textasciitilde{}/.s3ql/authinfo2)}
\item [-{-}debug \textless{}module\textgreater{}]
activate debugging output from \textless{}module\textgreater{}. Use \code{all} to
-get debug messages from all modules. This option can
-be specified multiple times.
+get debug messages from all modules. This option can be
+specified multiple times.
\item [-{-}quiet]
be really quiet
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
-\item [-{-}s3-location \textless{}name\textgreater{}]
-Storage location for new S3 buckets. Allowed values:
-\code{EU}, \code{us-west-1}, \code{ap-southeast-1}, or \code{us-standard}.
-(default: EU)
\item [-L \textless{}name\textgreater{}]
Filesystem label
\item [-{-}blocksize \textless{}size\textgreater{}]
@@ -663,10 +702,10 @@ Overwrite any existing data.
\end{optionlist}
\end{quote}
-Unless you have specified the \code{-{-}plain} option, \code{mkfs.s3ql} will ask you
-to enter an encryption password. If you do not want to enter this
-password every time that you mount the file system, you can store it
-in the \code{\textasciitilde{}/.s3ql/authinfo} file, see {\hyperref[mount:bucket-pw]{\emph{Storing Encryption Passwords}}}.
+Unless you have specified the \code{-{-}plain} option, \code{mkfs.s3ql} will ask
+you to enter an encryption password. This password will \emph{not} be read
+from an authentication file specified with the \code{-{-}authfile}
+option to prevent accidental creation of an encrypted bucket.
\chapter{Managing Buckets}
@@ -682,7 +721,7 @@ The syntax is
\end{Verbatim}
where \code{action} may be either of \textbf{passphrase},
-\textbf{upgrade}, \textbf{delete} or \textbf{download-metadata}.
+\textbf{upgrade}, \textbf{clear} or \textbf{download-metadata}.
The \textbf{s3qladm} accepts the following general options, no
matter what specific action is being invoked:
@@ -694,17 +733,18 @@ debug messages from all modules. This option can be
specified multiple times.
\item [-{-}quiet]
be really quiet
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication info.
-(default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}log \textless{}target\textgreater{}]
+Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify \code{none} to disable logging. Default:
+\code{none}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file (default:
+\code{\textasciitilde{}/.s3ql/authinfo2)}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default: \code{\textasciitilde{}/.s3ql)}
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
\end{optionlist}
\end{quote}
@@ -747,7 +787,7 @@ execute
A file system can be deleted with:
\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{s3qladm delete }\PYG{n+nv}{\textless{}storage url\textgreater{}}
+\PYG{l}{s3qladm clear }\PYG{n+nv}{\textless{}storage url\textgreater{}}
\end{Verbatim}
This physically deletes all the data and file system structures.
@@ -795,9 +835,17 @@ mounted on one computer at a time.
This command accepts the following options:
\begin{quote}
\begin{optionlist}{3cm}
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication
-info. (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}log \textless{}target\textgreater{}]
+Write logging info into this file. File will be
+rotated when it reaches 1 MB, and at most 5 old log
+files will be kept. Specify \code{none} to disable
+logging. Default: \code{\textasciitilde{}/.s3ql/mount.log}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default:
+\code{\textasciitilde{}/.s3ql)}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file
+(default: \code{\textasciitilde{}/.s3ql/authinfo2)}
\item [-{-}debug \textless{}module\textgreater{}]
activate debugging output from \textless{}module\textgreater{}. Use \code{all} to
get debug messages from all modules. This option can
@@ -806,12 +854,6 @@ be specified multiple times.
be really quiet
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
\item [-{-}cachesize \textless{}size\textgreater{}]
Cache size in kb (default: 102400 (100 MB)). Should be
at least 10 times the blocksize of the filesystem,
@@ -854,42 +896,16 @@ Allowed values: \code{lzma}, \code{bzip2}, \code{zlib}, none.
\item [-{-}metadata-upload-interval \textless{}seconds\textgreater{}]
Interval in seconds between complete metadata uploads.
Set to 0 to disable. Default: 24h.
-\item [-{-}compression-threads \textless{}no\textgreater{}]
-Number of parallel compression and encryption threads
-to use (default: 1).
+\item [-{-}threads \textless{}no\textgreater{}]
+Number of parallel upload threads to use (default:
+auto).
+\item [-{-}nfs]
+Support export of S3QL file systems over NFS (default:
+False)
\end{optionlist}
\end{quote}
-\section{Storing Encryption Passwords}
-\label{mount:bucket-pw}\label{mount:storing-encryption-passwords}
-If you are trying to mount an encrypted bucket, \code{mount.s3ql} will first
-try to read the password from the \code{.s3ql/authinfo} file (the same file
-that is used to read the backend authentication data) and prompt the
-user to enter the password only if this fails.
-
-The \code{authinfo} entries to specify bucket passwords are of the form
-
-\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{storage-url }\PYG{n+nv}{\textless{}storage-url\textgreater{}}\PYG{l}{ password }\PYG{n+nv}{\textless{}password\textgreater{}}
-\end{Verbatim}
-
-So to always use the password \code{topsecret} when mounting \code{s3://joes\_bucket},
-the entry would be
-
-\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{storage-url s3://joes\PYGZus{}bucket password topsecret}
-\end{Verbatim}
-
-\begin{notice}{note}{Note:}
-If you are using the local backend, the storage url will
-always be converted to an absolute path. So if you are in the
-\code{/home/john} directory and try to mount \code{local://bucket}, the matching
-\code{authinfo} entry has to have a storage url of
-\code{local:///home/john/bucket}.
-\end{notice}
-
-
\section{Compression Algorithms}
\label{mount:compression-algorithms}
S3QL supports three compression algorithms, LZMA, Bzip2 and zlib (with
@@ -921,9 +937,8 @@ videos at the same time).
\section{Parallel Compression}
\label{mount:parallel-compression}
If you are running S3QL on a system with multiple cores, you might
-want to set \code{-{-}compression-threads} to a value bigger than one. This
-will instruct S3QL to compress and encrypt several blocks at the same
-time.
+want to set the \code{-{-}threads} value larger than one. This will
+instruct S3QL to compress and encrypt several blocks at the same time.
If you want to do this in combination with using the LZMA compression
algorithm, you should keep an eye on memory usage though. Every
@@ -1045,8 +1060,8 @@ mounted.
\section{Snapshotting and Copy-on-Write}
\label{special:snapshotting-and-copy-on-write}\label{special:s3qlcp}
The command \code{s3qlcp} can be used to duplicate a directory tree without
-physically copying the file contents. This is possible due to the data
-de-duplication feature of S3QL.
+physically copying the file contents. This is made possible by the
+data de-duplication feature of S3QL.
The syntax of \code{s3qlcp} is:
@@ -1076,7 +1091,7 @@ snapshot of \code{\textless{}target\textgreater{}} or vice versa. However, the m
usage of \code{s3qlcp} is to regularly duplicate the same source
directory, say \code{documents}, to different target directories. For a
e.g. monthly replication, the target directories would typically be
-named something like \code{documents\_Januray} for the replication in
+named something like \code{documents\_January} for the replication in
January, \code{documents\_February} for the replication in February etc.
In this case it is clear that the target directories should be
regarded as snapshots of the source directory.
@@ -1088,12 +1103,6 @@ magnitude slower, because \code{cp} would have to read every file
completely (so that S3QL had to fetch all the data over the network
from the backend) before writing them into the destination folder.
-\item {}
-Before starting with the replication, S3QL has to flush the local
-cache. So if you just copied lots of new data into the file system
-that has not yet been uploaded, replication will take longer than
-usual.
-
\end{itemize}
@@ -1292,9 +1301,16 @@ or if you suspect that there might be errors, you should run the
This command accepts the following options:
\begin{quote}
\begin{optionlist}{3cm}
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication info.
-(default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}log \textless{}target\textgreater{}]
+Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify \code{none} to disable logging. Default:
+\code{\textasciitilde{}/.s3ql/fsck.log}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file (default:
+\code{\textasciitilde{}/.s3ql/authinfo2)}
\item [-{-}debug \textless{}module\textgreater{}]
activate debugging output from \textless{}module\textgreater{}. Use \code{all} to get
debug messages from all modules. This option can be
@@ -1303,12 +1319,6 @@ specified multiple times.
be really quiet
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext even
-for unencrypted file systems.
\item [-{-}batch]
If user input is required, exit without prompting.
\item [-{-}force]
@@ -1328,8 +1338,10 @@ the \code{contrib} directory of the source distribution or in
\section{benchmark.py}
\label{contrib:benchmark-py}
-This program measures your uplink bandwidth and compression speed and
-recommends a compression algorithm for optimal throughput.
+This program measures S3QL write performance, uplink bandwidth and
+compression speed to determine the limiting factor. It also gives
+recommendation for compression algorithm and number of upload threads
+to achieve maximum performance.
\section{s3\_copy.py}
@@ -1344,7 +1356,7 @@ migrate buckets to a different storage region or storage class
\code{pcp.py} is a wrapper program that starts several rsync processes to
copy directory trees in parallel. This is important because
transferring files in parallel significantly enhances performance when
-copying data from an S3QL file system (see {\hyperref[tips:copy-performance]{\emph{Permanently mounted backup file system}}} for
+copying data from an S3QL file system (see {\hyperref[tips:copy-performance]{\emph{Improving copy performance}}} for
details).
To recursively copy the directory \code{/mnt/home-backup} into
@@ -1478,8 +1490,25 @@ properly unmounts it when the system is shut down.
\chapter{Tips \& Tricks}
\label{tips:tips-tricks}\label{tips::doc}
+\section{SSH Backend}
+\label{tips:ssh-tipp}\label{tips:ssh-backend}
+By combining S3QL's local backend with \href{http://fuse.sourceforge.net/sshfs.html}{sshfs}, it is possible to store an
+S3QL file system on arbitrary SSH servers: first mount the remote
+target directory into the local filesystem,
+
+\begin{Verbatim}[commandchars=\\\{\}]
+\PYG{l}{sshfs user@my.server.com:/mnt/s3ql /mnt/sshfs}
+\end{Verbatim}
+
+and then give the mountpoint to S3QL as a local destination:
+
+\begin{Verbatim}[commandchars=\\\{\}]
+\PYG{l}{mount.s3ql local:///mnt/sshfs/mybucket /mnt/s3ql}
+\end{Verbatim}
+
+
\section{Permanently mounted backup file system}
-\label{tips:copy-performance}\label{tips:permanently-mounted-backup-file-system}
+\label{tips:permanently-mounted-backup-file-system}
If you use S3QL as a backup file system, it can be useful to mount the
file system permanently (rather than just mounting it for a backup and
unmounting it afterwards). Especially if your file system becomes
@@ -1503,7 +1532,12 @@ to zero).
\section{Improving copy performance}
-\label{tips:improving-copy-performance}
+\label{tips:copy-performance}\label{tips:improving-copy-performance}
+\begin{notice}{note}{Note:}
+The following applies only when copying data \textbf{from} an S3QL file
+system, \textbf{not} when copying data \textbf{to} an S3QL file system.
+\end{notice}
+
If you want to copy a lot of smaller files \emph{from} an S3QL file system
(e.g. for a system restore) you will probably notice that the
performance is rather bad.
@@ -1557,6 +1591,11 @@ details.
\chapter{Known Issues}
\label{issues:known-issues}\label{issues::doc}\begin{itemize}
\item {}
+S3QL does not verify TLS/SSL server certificates, so a
+man-in-the-middle attack is principally possible. See \href{http://code.google.com/p/s3ql/issues/detail?id=267}{issue 267} for more
+details.
+
+\item {}
S3QL is rather slow when an application tries to write data in
unreasonably small chunks. If a 1 MB file is copied in chunks of 1
KB, this will take more than 10 times as long as when it's copied
@@ -1641,7 +1680,7 @@ use a dedicated init script instead).
\item {}
S3QL relies on the backends not to run out of space. This is a given
for big storage providers like Amazon S3, but you may stumble upon
-this if you store buckets e.g. on a small sftp server.
+this if you store buckets e.g. on smaller servers or servies.
If there is no space left in the backend, attempts to write more
data into the S3QL file system will fail and the file system will be
@@ -1678,34 +1717,14 @@ here in the User's Guide.
\subsection{Description}
\label{man/mkfs:description}
The \textbf{mkfs.s3ql} command creates a new file system in the location
-specified by \emph{storage url}.
-
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-
-
-\subsubsection{Amazon S3}
-\label{man/mkfs:amazon-s3}
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \code{s3://\textless{}bucketname\textgreater{}}. Bucket names must conform to the S3 Bucket
-Name Restrictions.
+specified by \emph{storage url}. The storage url depends on the backend
+that is used. The S3QL User's Guide should be consulted for a
+description of the available backends.
-
-\subsubsection{Local}
-\label{man/mkfs:local}
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\code{local://\textless{}path\textgreater{}}. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \code{local:///var/archive}.
-
-
-\subsubsection{SFTP}
-\label{man/mkfs:sftp}
-The storage URL for SFTP connections has the form
-
-\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{sftp://}\PYG{n+nv}{\textless{}host\textgreater{}}\PYG{g+ge}{[:port]}\PYG{l}{/}\PYG{n+nv}{\textless{}path\textgreater{}}
-\end{Verbatim}
+Unless you have specified the \code{-{-}plain} option, \code{mkfs.s3ql} will ask
+you to enter an encryption password. This password will \emph{not} be read
+from an authentication file specified with the \code{-{-}authfile}
+option to prevent accidental creation of an encrypted bucket.
\subsection{Options}
@@ -1713,27 +1732,19 @@ The storage URL for SFTP connections has the form
The \textbf{mkfs.s3ql} command accepts the following options.
\begin{quote}
\begin{optionlist}{3cm}
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication
-info. (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file (default:
+\code{\textasciitilde{}/.s3ql/authinfo2)}
\item [-{-}debug \textless{}module\textgreater{}]
activate debugging output from \textless{}module\textgreater{}. Use \code{all} to
-get debug messages from all modules. This option can
-be specified multiple times.
+get debug messages from all modules. This option can be
+specified multiple times.
\item [-{-}quiet]
be really quiet
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
-\item [-{-}s3-location \textless{}name\textgreater{}]
-Storage location for new S3 buckets. Allowed values:
-\code{EU}, \code{us-west-1}, \code{ap-southeast-1}, or \code{us-standard}.
-(default: EU)
\item [-L \textless{}name\textgreater{}]
Filesystem label
\item [-{-}blocksize \textless{}size\textgreater{}]
@@ -1746,14 +1757,6 @@ Overwrite any existing data.
\end{quote}
-\subsection{Files}
-\label{man/mkfs:files}
-Authentication data for backends and bucket encryption passphrases are
-read from \code{authinfo} in \code{\textasciitilde{}/.s3ql} or the directory
-specified with \code{-{-}homedir}. Log files are placed in the same
-directory.
-
-
\subsection{Exit Status}
\label{man/mkfs:exit-status}
\textbf{mkfs.s3ql} returns exit code 0 if the operation succeeded and 1 if some
@@ -1788,32 +1791,8 @@ The \textbf{s3qladm} command performs various operations on S3QL buckets.
The file system contained in the bucket \emph{must not be mounted} when
using \textbf{s3qladm} or things will go wrong badly.
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-
-
-\subsubsection{Amazon S3}
-\label{man/adm:amazon-s3}
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \code{s3://\textless{}bucketname\textgreater{}}. Bucket names must conform to the S3 Bucket
-Name Restrictions.
-
-
-\subsubsection{Local}
-\label{man/adm:local}
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\code{local://\textless{}path\textgreater{}}. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \code{local:///var/archive}.
-
-
-\subsubsection{SFTP}
-\label{man/adm:sftp}
-The storage URL for SFTP connections has the form
-
-\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{sftp://}\PYG{n+nv}{\textless{}host\textgreater{}}\PYG{g+ge}{[:port]}\PYG{l}{/}\PYG{n+nv}{\textless{}path\textgreater{}}
-\end{Verbatim}
+The storage url depends on the backend that is used. The S3QL User's
+Guide should be consulted for a description of the available backends.
\subsection{Options}
@@ -1827,17 +1806,18 @@ debug messages from all modules. This option can be
specified multiple times.
\item [-{-}quiet]
be really quiet
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication info.
-(default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}log \textless{}target\textgreater{}]
+Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify \code{none} to disable logging. Default:
+\code{none}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file (default:
+\code{\textasciitilde{}/.s3ql/authinfo2)}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default: \code{\textasciitilde{}/.s3ql)}
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
\end{optionlist}
\end{quote}
@@ -1864,14 +1844,6 @@ Interactively download backups of the file system metadata.
\end{description}
-\subsection{Files}
-\label{man/adm:files}
-Authentication data for backends and bucket encryption passphrases are
-read from \code{authinfo} in \code{\textasciitilde{}/.s3ql} or the directory
-specified with \code{-{-}homedir}. Log files are placed in the same
-directory.
-
-
\subsection{Exit Status}
\label{man/adm:exit-status}
\textbf{s3qladm} returns exit code 0 if the operation succeeded and 1 if some
@@ -1900,34 +1872,9 @@ system, conventional locations are \code{/usr/share/doc/s3ql} or
\subsection{Description}
\label{man/mount:description}
The \textbf{mount.s3ql} command mounts the S3QL file system stored in \emph{storage
-url} in the directory \emph{mount point}.
-
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-
-
-\subsubsection{Amazon S3}
-\label{man/mount:amazon-s3}
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \code{s3://\textless{}bucketname\textgreater{}}. Bucket names must conform to the S3 Bucket
-Name Restrictions.
-
-
-\subsubsection{Local}
-\label{man/mount:local}
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\code{local://\textless{}path\textgreater{}}. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \code{local:///var/archive}.
-
-
-\subsubsection{SFTP}
-\label{man/mount:sftp}
-The storage URL for SFTP connections has the form
-
-\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{sftp://}\PYG{n+nv}{\textless{}host\textgreater{}}\PYG{g+ge}{[:port]}\PYG{l}{/}\PYG{n+nv}{\textless{}path\textgreater{}}
-\end{Verbatim}
+url} in the directory \emph{mount point}. The storage url depends on the
+backend that is used. The S3QL User's Guide should be consulted for a
+description of the available backends.
\subsection{Options}
@@ -1935,9 +1882,17 @@ The storage URL for SFTP connections has the form
The \textbf{mount.s3ql} command accepts the following options.
\begin{quote}
\begin{optionlist}{3cm}
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication
-info. (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}log \textless{}target\textgreater{}]
+Write logging info into this file. File will be
+rotated when it reaches 1 MB, and at most 5 old log
+files will be kept. Specify \code{none} to disable
+logging. Default: \code{\textasciitilde{}/.s3ql/mount.log}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default:
+\code{\textasciitilde{}/.s3ql)}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file
+(default: \code{\textasciitilde{}/.s3ql/authinfo2)}
\item [-{-}debug \textless{}module\textgreater{}]
activate debugging output from \textless{}module\textgreater{}. Use \code{all} to
get debug messages from all modules. This option can
@@ -1946,12 +1901,6 @@ be specified multiple times.
be really quiet
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
\item [-{-}cachesize \textless{}size\textgreater{}]
Cache size in kb (default: 102400 (100 MB)). Should be
at least 10 times the blocksize of the filesystem,
@@ -1994,21 +1943,16 @@ Allowed values: \code{lzma}, \code{bzip2}, \code{zlib}, none.
\item [-{-}metadata-upload-interval \textless{}seconds\textgreater{}]
Interval in seconds between complete metadata uploads.
Set to 0 to disable. Default: 24h.
-\item [-{-}compression-threads \textless{}no\textgreater{}]
-Number of parallel compression and encryption threads
-to use (default: 1).
+\item [-{-}threads \textless{}no\textgreater{}]
+Number of parallel upload threads to use (default:
+auto).
+\item [-{-}nfs]
+Support export of S3QL file systems over NFS (default:
+False)
\end{optionlist}
\end{quote}
-\subsection{Files}
-\label{man/mount:files}
-Authentication data for backends and bucket encryption passphrases are
-read from \code{authinfo} in \code{\textasciitilde{}/.s3ql} or the directory
-specified with \code{-{-}homedir}. Log files are placed in the same
-directory.
-
-
\subsection{Exit Status}
\label{man/mount:exit-status}
\textbf{mount.s3ql} returns exit code 0 if the operation succeeded and 1 if some
@@ -2094,6 +2038,11 @@ where \code{action} may be either of \textbf{flushcache},
The \textbf{s3qlctrl} command performs various actions on the S3QL file system mounted
in \code{mountpoint}.
+\textbf{s3qlctrl} can only be called by the user that mounted the file system
+and (if the file system was mounted with \code{-{-}allow-other} or
+\code{-{-}allow-root}) the root user. This limitation might be
+removed in the future (see \href{http://code.google.com/p/s3ql/issues/detail?id=155}{issue 155}).
+
The following actions may be specified:
\begin{description}
\item[{flushcache}] \leavevmode
@@ -2195,7 +2144,7 @@ snapshot of \code{\textless{}target\textgreater{}} or vice versa. However, the m
usage of \code{s3qlcp} is to regularly duplicate the same source
directory, say \code{documents}, to different target directories. For a
e.g. monthly replication, the target directories would typically be
-named something like \code{documents\_Januray} for the replication in
+named something like \code{documents\_January} for the replication in
January, \code{documents\_February} for the replication in February etc.
In this case it is clear that the target directories should be
regarded as snapshots of the source directory.
@@ -2207,12 +2156,6 @@ magnitude slower, because \code{cp} would have to read every file
completely (so that S3QL had to fetch all the data over the network
from the backend) before writing them into the destination folder.
-\item {}
-Before starting with the replication, S3QL has to flush the local
-cache. So if you just copied lots of new data into the file system
-that has not yet been uploaded, replication will take longer than
-usual.
-
\end{itemize}
@@ -2293,6 +2236,11 @@ you to delete immutable trees (which can be created with
Be warned that there is no additional confirmation. The directory will
be removed entirely and immediately.
+\textbf{s3qlrm} can only be called by the user that mounted the file system
+and (if the file system was mounted with \code{-{-}allow-other} or
+\code{-{-}allow-root}) the root user. This limitation might be
+removed in the future (see \href{http://code.google.com/p/s3ql/issues/detail?id=155}{issue 155}).
+
\subsection{Options}
\label{man/rm:options}
@@ -2342,6 +2290,11 @@ whatsoever. You can not add new files or directories and you can not
change or delete existing files and directories. The only way to get
rid of an immutable tree is to use the \textbf{s3qlrm} command.
+\textbf{s3qllock} can only be called by the user that mounted the file system
+and (if the file system was mounted with \code{-{-}allow-other} or
+\code{-{-}allow-root}) the root user. This limitation might be
+removed in the future (see \href{http://code.google.com/p/s3ql/issues/detail?id=155}{issue 155}).
+
\subsection{Rationale}
\label{man/lock:rationale}
@@ -2480,34 +2433,9 @@ system, conventional locations are \code{/usr/share/doc/s3ql} or
\label{man/fsck:description}
The \textbf{mkfs.s3ql} command checks the new file system in the location
specified by \emph{storage url} for errors and attempts to repair any
-problems.
-
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-
-
-\subsubsection{Amazon S3}
-\label{man/fsck:amazon-s3}
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \code{s3://\textless{}bucketname\textgreater{}}. Bucket names must conform to the S3 Bucket
-Name Restrictions.
-
-
-\subsubsection{Local}
-\label{man/fsck:local}
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\code{local://\textless{}path\textgreater{}}. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \code{local:///var/archive}.
-
-
-\subsubsection{SFTP}
-\label{man/fsck:sftp}
-The storage URL for SFTP connections has the form
-
-\begin{Verbatim}[commandchars=\\\{\}]
-\PYG{l}{sftp://}\PYG{n+nv}{\textless{}host\textgreater{}}\PYG{g+ge}{[:port]}\PYG{l}{/}\PYG{n+nv}{\textless{}path\textgreater{}}
-\end{Verbatim}
+problems. The storage url depends on the backend that is used. The
+S3QL User's Guide should be consulted for a description of the
+available backends.
\subsection{Options}
@@ -2515,9 +2443,16 @@ The storage URL for SFTP connections has the form
The \textbf{mkfs.s3ql} command accepts the following options.
\begin{quote}
\begin{optionlist}{3cm}
-\item [-{-}homedir \textless{}path\textgreater{}]
-Directory for log files, cache and authentication info.
-(default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}log \textless{}target\textgreater{}]
+Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify \code{none} to disable logging. Default:
+\code{\textasciitilde{}/.s3ql/fsck.log}
+\item [-{-}cachedir \textless{}path\textgreater{}]
+Store cached data in this directory (default: \code{\textasciitilde{}/.s3ql)}
+\item [-{-}authfile \textless{}path\textgreater{}]
+Read authentication credentials from this file (default:
+\code{\textasciitilde{}/.s3ql/authinfo2)}
\item [-{-}debug \textless{}module\textgreater{}]
activate debugging output from \textless{}module\textgreater{}. Use \code{all} to get
debug messages from all modules. This option can be
@@ -2526,12 +2461,6 @@ specified multiple times.
be really quiet
\item [-{-}version]
just print program version and exit
-\item [-{-}ssl]
-Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext even
-for unencrypted file systems.
\item [-{-}batch]
If user input is required, exit without prompting.
\item [-{-}force]
@@ -2540,14 +2469,6 @@ Force checking even if file system is marked clean.
\end{quote}
-\subsection{Files}
-\label{man/fsck:files}
-Authentication data for backends and bucket encryption passphrases are
-read from \code{authinfo} in \code{\textasciitilde{}/.s3ql} or the directory
-specified with \code{-{-}homedir}. Log files are placed in the same
-directory.
-
-
\subsection{Exit Status}
\label{man/fsck:exit-status}
\textbf{mkfs.s3ql} returns exit code 0 if the operation succeeded and 1 if some
@@ -2580,6 +2501,10 @@ The \textbf{pcp} command is a is a wrapper that starts several
allows much better copying performance on file system that have
relatively high latency when retrieving individual files like S3QL.
+\textbf{Note}: Using this program only improves performance when copying
+\emph{from} an S3QL file system. When copying \emph{to} an S3QL file system,
+using \textbf{pcp} is more likely to \emph{decrease} performance.
+
\subsection{Options}
\label{man/pcp:options}
diff --git a/doc/latex/manual.toc b/doc/latex/manual.toc
index 2996158..d52b4fc 100644
--- a/doc/latex/manual.toc
+++ b/doc/latex/manual.toc
@@ -5,138 +5,126 @@
\contentsline {chapter}{\numberline {2}Installation}{3}{chapter.2}
\contentsline {section}{\numberline {2.1}Dependencies}{3}{section.2.1}
\contentsline {section}{\numberline {2.2}Installing S3QL}{4}{section.2.2}
-\contentsline {chapter}{\numberline {3}Storage Backends}{5}{chapter.3}
-\contentsline {section}{\numberline {3.1}On Backend Reliability}{5}{section.3.1}
-\contentsline {section}{\numberline {3.2}The \texttt {authinfo} file}{6}{section.3.2}
-\contentsline {section}{\numberline {3.3}Consistency Guarantees}{6}{section.3.3}
-\contentsline {subsection}{\numberline {3.3.1}Dealing with Eventual Consistency}{6}{subsection.3.3.1}
-\contentsline {section}{\numberline {3.4}The Amazon S3 Backend}{7}{section.3.4}
-\contentsline {section}{\numberline {3.5}The Local Backend}{8}{section.3.5}
-\contentsline {section}{\numberline {3.6}The SFTP Backend}{8}{section.3.6}
-\contentsline {chapter}{\numberline {4}File System Creation}{9}{chapter.4}
-\contentsline {chapter}{\numberline {5}Managing Buckets}{11}{chapter.5}
-\contentsline {section}{\numberline {5.1}Changing the Passphrase}{11}{section.5.1}
-\contentsline {section}{\numberline {5.2}Upgrading the file system}{11}{section.5.2}
-\contentsline {section}{\numberline {5.3}Deleting a file system}{12}{section.5.3}
-\contentsline {section}{\numberline {5.4}Restoring Metadata Backups}{12}{section.5.4}
-\contentsline {chapter}{\numberline {6}Mounting}{13}{chapter.6}
-\contentsline {section}{\numberline {6.1}Storing Encryption Passwords}{14}{section.6.1}
-\contentsline {section}{\numberline {6.2}Compression Algorithms}{14}{section.6.2}
-\contentsline {section}{\numberline {6.3}Parallel Compression}{15}{section.6.3}
-\contentsline {section}{\numberline {6.4}Notes about Caching}{15}{section.6.4}
-\contentsline {subsection}{\numberline {6.4.1}Maximum Number of Cache Entries}{15}{subsection.6.4.1}
-\contentsline {subsection}{\numberline {6.4.2}Cache Flushing and Expiration}{15}{subsection.6.4.2}
-\contentsline {section}{\numberline {6.5}Automatic Mounting}{15}{section.6.5}
-\contentsline {chapter}{\numberline {7}Advanced S3QL Features}{17}{chapter.7}
-\contentsline {section}{\numberline {7.1}Snapshotting and Copy-on-Write}{17}{section.7.1}
-\contentsline {subsection}{\numberline {7.1.1}Snapshotting vs Hardlinking}{17}{subsection.7.1.1}
-\contentsline {section}{\numberline {7.2}Getting Statistics}{18}{section.7.2}
-\contentsline {section}{\numberline {7.3}Immutable Trees}{18}{section.7.3}
-\contentsline {section}{\numberline {7.4}Fast Recursive Removal}{19}{section.7.4}
-\contentsline {section}{\numberline {7.5}Runtime Configuration}{19}{section.7.5}
-\contentsline {chapter}{\numberline {8}Unmounting}{21}{chapter.8}
-\contentsline {chapter}{\numberline {9}Checking for Errors}{23}{chapter.9}
-\contentsline {chapter}{\numberline {10}Contributed Programs}{25}{chapter.10}
-\contentsline {section}{\numberline {10.1}benchmark.py}{25}{section.10.1}
-\contentsline {section}{\numberline {10.2}s3\_copy.py}{25}{section.10.2}
-\contentsline {section}{\numberline {10.3}pcp.py}{25}{section.10.3}
-\contentsline {section}{\numberline {10.4}s3\_backup.sh}{25}{section.10.4}
-\contentsline {section}{\numberline {10.5}expire\_backups.py}{26}{section.10.5}
-\contentsline {section}{\numberline {10.6}s3ql.conf}{27}{section.10.6}
-\contentsline {chapter}{\numberline {11}Tips \& Tricks}{29}{chapter.11}
-\contentsline {section}{\numberline {11.1}Permanently mounted backup file system}{29}{section.11.1}
-\contentsline {section}{\numberline {11.2}Improving copy performance}{29}{section.11.2}
-\contentsline {chapter}{\numberline {12}Known Issues}{31}{chapter.12}
-\contentsline {chapter}{\numberline {13}Manpages}{33}{chapter.13}
-\contentsline {section}{\numberline {13.1}The \textbf {mkfs.s3ql} command}{33}{section.13.1}
-\contentsline {subsection}{\numberline {13.1.1}Synopsis}{33}{subsection.13.1.1}
-\contentsline {subsection}{\numberline {13.1.2}Description}{33}{subsection.13.1.2}
-\contentsline {subsubsection}{Amazon S3}{33}{subsubsection*.3}
-\contentsline {subsubsection}{Local}{33}{subsubsection*.4}
-\contentsline {subsubsection}{SFTP}{33}{subsubsection*.5}
-\contentsline {subsection}{\numberline {13.1.3}Options}{34}{subsection.13.1.3}
-\contentsline {subsection}{\numberline {13.1.4}Files}{34}{subsection.13.1.4}
-\contentsline {subsection}{\numberline {13.1.5}Exit Status}{34}{subsection.13.1.5}
-\contentsline {subsection}{\numberline {13.1.6}See Also}{34}{subsection.13.1.6}
-\contentsline {section}{\numberline {13.2}The \textbf {s3qladm} command}{34}{section.13.2}
-\contentsline {subsection}{\numberline {13.2.1}Synopsis}{34}{subsection.13.2.1}
-\contentsline {subsection}{\numberline {13.2.2}Description}{35}{subsection.13.2.2}
-\contentsline {subsubsection}{Amazon S3}{35}{subsubsection*.6}
-\contentsline {subsubsection}{Local}{35}{subsubsection*.7}
-\contentsline {subsubsection}{SFTP}{35}{subsubsection*.8}
-\contentsline {subsection}{\numberline {13.2.3}Options}{35}{subsection.13.2.3}
-\contentsline {subsection}{\numberline {13.2.4}Actions}{35}{subsection.13.2.4}
-\contentsline {subsection}{\numberline {13.2.5}Files}{36}{subsection.13.2.5}
-\contentsline {subsection}{\numberline {13.2.6}Exit Status}{36}{subsection.13.2.6}
-\contentsline {subsection}{\numberline {13.2.7}See Also}{36}{subsection.13.2.7}
-\contentsline {section}{\numberline {13.3}The \textbf {mount.s3ql} command}{36}{section.13.3}
-\contentsline {subsection}{\numberline {13.3.1}Synopsis}{36}{subsection.13.3.1}
-\contentsline {subsection}{\numberline {13.3.2}Description}{36}{subsection.13.3.2}
-\contentsline {subsubsection}{Amazon S3}{36}{subsubsection*.9}
-\contentsline {subsubsection}{Local}{36}{subsubsection*.10}
-\contentsline {subsubsection}{SFTP}{36}{subsubsection*.11}
-\contentsline {subsection}{\numberline {13.3.3}Options}{37}{subsection.13.3.3}
-\contentsline {subsection}{\numberline {13.3.4}Files}{38}{subsection.13.3.4}
-\contentsline {subsection}{\numberline {13.3.5}Exit Status}{38}{subsection.13.3.5}
-\contentsline {subsection}{\numberline {13.3.6}See Also}{38}{subsection.13.3.6}
-\contentsline {section}{\numberline {13.4}The \textbf {s3qlstat} command}{38}{section.13.4}
-\contentsline {subsection}{\numberline {13.4.1}Synopsis}{38}{subsection.13.4.1}
-\contentsline {subsection}{\numberline {13.4.2}Description}{38}{subsection.13.4.2}
-\contentsline {subsection}{\numberline {13.4.3}Options}{38}{subsection.13.4.3}
-\contentsline {subsection}{\numberline {13.4.4}Exit Status}{38}{subsection.13.4.4}
-\contentsline {subsection}{\numberline {13.4.5}See Also}{38}{subsection.13.4.5}
-\contentsline {section}{\numberline {13.5}The \textbf {s3qlctrl} command}{39}{section.13.5}
-\contentsline {subsection}{\numberline {13.5.1}Synopsis}{39}{subsection.13.5.1}
-\contentsline {subsection}{\numberline {13.5.2}Description}{39}{subsection.13.5.2}
-\contentsline {subsection}{\numberline {13.5.3}Options}{39}{subsection.13.5.3}
-\contentsline {subsection}{\numberline {13.5.4}Exit Status}{39}{subsection.13.5.4}
-\contentsline {subsection}{\numberline {13.5.5}See Also}{39}{subsection.13.5.5}
-\contentsline {section}{\numberline {13.6}The \textbf {s3qlcp} command}{40}{section.13.6}
-\contentsline {subsection}{\numberline {13.6.1}Synopsis}{40}{subsection.13.6.1}
-\contentsline {subsection}{\numberline {13.6.2}Description}{40}{subsection.13.6.2}
-\contentsline {subsubsection}{Snapshotting vs Hardlinking}{40}{subsubsection*.12}
-\contentsline {subsection}{\numberline {13.6.3}Options}{40}{subsection.13.6.3}
-\contentsline {subsection}{\numberline {13.6.4}Exit Status}{41}{subsection.13.6.4}
-\contentsline {subsection}{\numberline {13.6.5}See Also}{41}{subsection.13.6.5}
-\contentsline {section}{\numberline {13.7}The \textbf {s3qlrm} command}{41}{section.13.7}
-\contentsline {subsection}{\numberline {13.7.1}Synopsis}{41}{subsection.13.7.1}
-\contentsline {subsection}{\numberline {13.7.2}Description}{41}{subsection.13.7.2}
-\contentsline {subsection}{\numberline {13.7.3}Options}{41}{subsection.13.7.3}
-\contentsline {subsection}{\numberline {13.7.4}Exit Status}{41}{subsection.13.7.4}
-\contentsline {subsection}{\numberline {13.7.5}See Also}{41}{subsection.13.7.5}
-\contentsline {section}{\numberline {13.8}The \textbf {s3qllock} command}{42}{section.13.8}
-\contentsline {subsection}{\numberline {13.8.1}Synopsis}{42}{subsection.13.8.1}
-\contentsline {subsection}{\numberline {13.8.2}Description}{42}{subsection.13.8.2}
-\contentsline {subsection}{\numberline {13.8.3}Rationale}{42}{subsection.13.8.3}
-\contentsline {subsection}{\numberline {13.8.4}Options}{42}{subsection.13.8.4}
-\contentsline {subsection}{\numberline {13.8.5}Exit Status}{42}{subsection.13.8.5}
-\contentsline {subsection}{\numberline {13.8.6}See Also}{43}{subsection.13.8.6}
-\contentsline {section}{\numberline {13.9}The \textbf {umount.s3ql} command}{43}{section.13.9}
-\contentsline {subsection}{\numberline {13.9.1}Synopsis}{43}{subsection.13.9.1}
-\contentsline {subsection}{\numberline {13.9.2}Description}{43}{subsection.13.9.2}
-\contentsline {subsection}{\numberline {13.9.3}Options}{43}{subsection.13.9.3}
-\contentsline {subsection}{\numberline {13.9.4}Exit Status}{43}{subsection.13.9.4}
-\contentsline {subsection}{\numberline {13.9.5}See Also}{43}{subsection.13.9.5}
-\contentsline {section}{\numberline {13.10}The \textbf {fsck.s3ql} command}{44}{section.13.10}
-\contentsline {subsection}{\numberline {13.10.1}Synopsis}{44}{subsection.13.10.1}
-\contentsline {subsection}{\numberline {13.10.2}Description}{44}{subsection.13.10.2}
-\contentsline {subsubsection}{Amazon S3}{44}{subsubsection*.13}
-\contentsline {subsubsection}{Local}{44}{subsubsection*.14}
-\contentsline {subsubsection}{SFTP}{44}{subsubsection*.15}
-\contentsline {subsection}{\numberline {13.10.3}Options}{44}{subsection.13.10.3}
-\contentsline {subsection}{\numberline {13.10.4}Files}{45}{subsection.13.10.4}
-\contentsline {subsection}{\numberline {13.10.5}Exit Status}{45}{subsection.13.10.5}
-\contentsline {subsection}{\numberline {13.10.6}See Also}{45}{subsection.13.10.6}
-\contentsline {section}{\numberline {13.11}The \textbf {pcp} command}{45}{section.13.11}
-\contentsline {subsection}{\numberline {13.11.1}Synopsis}{45}{subsection.13.11.1}
-\contentsline {subsection}{\numberline {13.11.2}Description}{45}{subsection.13.11.2}
-\contentsline {subsection}{\numberline {13.11.3}Options}{45}{subsection.13.11.3}
-\contentsline {subsection}{\numberline {13.11.4}Exit Status}{45}{subsection.13.11.4}
-\contentsline {subsection}{\numberline {13.11.5}See Also}{45}{subsection.13.11.5}
-\contentsline {section}{\numberline {13.12}The \textbf {expire\_backups} command}{46}{section.13.12}
-\contentsline {subsection}{\numberline {13.12.1}Synopsis}{46}{subsection.13.12.1}
-\contentsline {subsection}{\numberline {13.12.2}Description}{46}{subsection.13.12.2}
-\contentsline {subsection}{\numberline {13.12.3}Options}{47}{subsection.13.12.3}
-\contentsline {subsection}{\numberline {13.12.4}Exit Status}{47}{subsection.13.12.4}
-\contentsline {subsection}{\numberline {13.12.5}See Also}{47}{subsection.13.12.5}
-\contentsline {chapter}{\numberline {14}Further Resources / Getting Help}{49}{chapter.14}
+\contentsline {chapter}{\numberline {3}General Information}{5}{chapter.3}
+\contentsline {section}{\numberline {3.1}Terminology}{5}{section.3.1}
+\contentsline {section}{\numberline {3.2}Storing Authentication Information}{5}{section.3.2}
+\contentsline {section}{\numberline {3.3}On Backend Reliability}{6}{section.3.3}
+\contentsline {chapter}{\numberline {4}Storage Backends}{9}{chapter.4}
+\contentsline {section}{\numberline {4.1}Google Storage}{9}{section.4.1}
+\contentsline {section}{\numberline {4.2}Amazon S3}{9}{section.4.2}
+\contentsline {subsection}{\numberline {4.2.1}Reduced Redundancy Storage (RRS)}{10}{subsection.4.2.1}
+\contentsline {subsection}{\numberline {4.2.2}Potential issues when using the US Standard storage region}{10}{subsection.4.2.2}
+\contentsline {section}{\numberline {4.3}S3 compatible}{10}{section.4.3}
+\contentsline {section}{\numberline {4.4}Local}{11}{section.4.4}
+\contentsline {section}{\numberline {4.5}SSH/SFTP}{11}{section.4.5}
+\contentsline {chapter}{\numberline {5}File System Creation}{13}{chapter.5}
+\contentsline {chapter}{\numberline {6}Managing Buckets}{15}{chapter.6}
+\contentsline {section}{\numberline {6.1}Changing the Passphrase}{15}{section.6.1}
+\contentsline {section}{\numberline {6.2}Upgrading the file system}{15}{section.6.2}
+\contentsline {section}{\numberline {6.3}Deleting a file system}{16}{section.6.3}
+\contentsline {section}{\numberline {6.4}Restoring Metadata Backups}{16}{section.6.4}
+\contentsline {chapter}{\numberline {7}Mounting}{17}{chapter.7}
+\contentsline {section}{\numberline {7.1}Compression Algorithms}{18}{section.7.1}
+\contentsline {section}{\numberline {7.2}Parallel Compression}{18}{section.7.2}
+\contentsline {section}{\numberline {7.3}Notes about Caching}{19}{section.7.3}
+\contentsline {subsection}{\numberline {7.3.1}Maximum Number of Cache Entries}{19}{subsection.7.3.1}
+\contentsline {subsection}{\numberline {7.3.2}Cache Flushing and Expiration}{19}{subsection.7.3.2}
+\contentsline {section}{\numberline {7.4}Automatic Mounting}{19}{section.7.4}
+\contentsline {chapter}{\numberline {8}Advanced S3QL Features}{21}{chapter.8}
+\contentsline {section}{\numberline {8.1}Snapshotting and Copy-on-Write}{21}{section.8.1}
+\contentsline {subsection}{\numberline {8.1.1}Snapshotting vs Hardlinking}{21}{subsection.8.1.1}
+\contentsline {section}{\numberline {8.2}Getting Statistics}{22}{section.8.2}
+\contentsline {section}{\numberline {8.3}Immutable Trees}{22}{section.8.3}
+\contentsline {section}{\numberline {8.4}Fast Recursive Removal}{23}{section.8.4}
+\contentsline {section}{\numberline {8.5}Runtime Configuration}{23}{section.8.5}
+\contentsline {chapter}{\numberline {9}Unmounting}{25}{chapter.9}
+\contentsline {chapter}{\numberline {10}Checking for Errors}{27}{chapter.10}
+\contentsline {chapter}{\numberline {11}Contributed Programs}{29}{chapter.11}
+\contentsline {section}{\numberline {11.1}benchmark.py}{29}{section.11.1}
+\contentsline {section}{\numberline {11.2}s3\_copy.py}{29}{section.11.2}
+\contentsline {section}{\numberline {11.3}pcp.py}{29}{section.11.3}
+\contentsline {section}{\numberline {11.4}s3\_backup.sh}{29}{section.11.4}
+\contentsline {section}{\numberline {11.5}expire\_backups.py}{30}{section.11.5}
+\contentsline {section}{\numberline {11.6}s3ql.conf}{31}{section.11.6}
+\contentsline {chapter}{\numberline {12}Tips \& Tricks}{33}{chapter.12}
+\contentsline {section}{\numberline {12.1}SSH Backend}{33}{section.12.1}
+\contentsline {section}{\numberline {12.2}Permanently mounted backup file system}{33}{section.12.2}
+\contentsline {section}{\numberline {12.3}Improving copy performance}{33}{section.12.3}
+\contentsline {chapter}{\numberline {13}Known Issues}{35}{chapter.13}
+\contentsline {chapter}{\numberline {14}Manpages}{37}{chapter.14}
+\contentsline {section}{\numberline {14.1}The \textbf {mkfs.s3ql} command}{37}{section.14.1}
+\contentsline {subsection}{\numberline {14.1.1}Synopsis}{37}{subsection.14.1.1}
+\contentsline {subsection}{\numberline {14.1.2}Description}{37}{subsection.14.1.2}
+\contentsline {subsection}{\numberline {14.1.3}Options}{37}{subsection.14.1.3}
+\contentsline {subsection}{\numberline {14.1.4}Exit Status}{38}{subsection.14.1.4}
+\contentsline {subsection}{\numberline {14.1.5}See Also}{38}{subsection.14.1.5}
+\contentsline {section}{\numberline {14.2}The \textbf {s3qladm} command}{38}{section.14.2}
+\contentsline {subsection}{\numberline {14.2.1}Synopsis}{38}{subsection.14.2.1}
+\contentsline {subsection}{\numberline {14.2.2}Description}{38}{subsection.14.2.2}
+\contentsline {subsection}{\numberline {14.2.3}Options}{38}{subsection.14.2.3}
+\contentsline {subsection}{\numberline {14.2.4}Actions}{39}{subsection.14.2.4}
+\contentsline {subsection}{\numberline {14.2.5}Exit Status}{39}{subsection.14.2.5}
+\contentsline {subsection}{\numberline {14.2.6}See Also}{39}{subsection.14.2.6}
+\contentsline {section}{\numberline {14.3}The \textbf {mount.s3ql} command}{39}{section.14.3}
+\contentsline {subsection}{\numberline {14.3.1}Synopsis}{39}{subsection.14.3.1}
+\contentsline {subsection}{\numberline {14.3.2}Description}{39}{subsection.14.3.2}
+\contentsline {subsection}{\numberline {14.3.3}Options}{39}{subsection.14.3.3}
+\contentsline {subsection}{\numberline {14.3.4}Exit Status}{40}{subsection.14.3.4}
+\contentsline {subsection}{\numberline {14.3.5}See Also}{40}{subsection.14.3.5}
+\contentsline {section}{\numberline {14.4}The \textbf {s3qlstat} command}{41}{section.14.4}
+\contentsline {subsection}{\numberline {14.4.1}Synopsis}{41}{subsection.14.4.1}
+\contentsline {subsection}{\numberline {14.4.2}Description}{41}{subsection.14.4.2}
+\contentsline {subsection}{\numberline {14.4.3}Options}{41}{subsection.14.4.3}
+\contentsline {subsection}{\numberline {14.4.4}Exit Status}{41}{subsection.14.4.4}
+\contentsline {subsection}{\numberline {14.4.5}See Also}{41}{subsection.14.4.5}
+\contentsline {section}{\numberline {14.5}The \textbf {s3qlctrl} command}{41}{section.14.5}
+\contentsline {subsection}{\numberline {14.5.1}Synopsis}{41}{subsection.14.5.1}
+\contentsline {subsection}{\numberline {14.5.2}Description}{41}{subsection.14.5.2}
+\contentsline {subsection}{\numberline {14.5.3}Options}{42}{subsection.14.5.3}
+\contentsline {subsection}{\numberline {14.5.4}Exit Status}{42}{subsection.14.5.4}
+\contentsline {subsection}{\numberline {14.5.5}See Also}{42}{subsection.14.5.5}
+\contentsline {section}{\numberline {14.6}The \textbf {s3qlcp} command}{42}{section.14.6}
+\contentsline {subsection}{\numberline {14.6.1}Synopsis}{42}{subsection.14.6.1}
+\contentsline {subsection}{\numberline {14.6.2}Description}{42}{subsection.14.6.2}
+\contentsline {subsubsection}{Snapshotting vs Hardlinking}{43}{subsubsection*.3}
+\contentsline {subsection}{\numberline {14.6.3}Options}{43}{subsection.14.6.3}
+\contentsline {subsection}{\numberline {14.6.4}Exit Status}{43}{subsection.14.6.4}
+\contentsline {subsection}{\numberline {14.6.5}See Also}{43}{subsection.14.6.5}
+\contentsline {section}{\numberline {14.7}The \textbf {s3qlrm} command}{44}{section.14.7}
+\contentsline {subsection}{\numberline {14.7.1}Synopsis}{44}{subsection.14.7.1}
+\contentsline {subsection}{\numberline {14.7.2}Description}{44}{subsection.14.7.2}
+\contentsline {subsection}{\numberline {14.7.3}Options}{44}{subsection.14.7.3}
+\contentsline {subsection}{\numberline {14.7.4}Exit Status}{44}{subsection.14.7.4}
+\contentsline {subsection}{\numberline {14.7.5}See Also}{44}{subsection.14.7.5}
+\contentsline {section}{\numberline {14.8}The \textbf {s3qllock} command}{44}{section.14.8}
+\contentsline {subsection}{\numberline {14.8.1}Synopsis}{44}{subsection.14.8.1}
+\contentsline {subsection}{\numberline {14.8.2}Description}{45}{subsection.14.8.2}
+\contentsline {subsection}{\numberline {14.8.3}Rationale}{45}{subsection.14.8.3}
+\contentsline {subsection}{\numberline {14.8.4}Options}{45}{subsection.14.8.4}
+\contentsline {subsection}{\numberline {14.8.5}Exit Status}{45}{subsection.14.8.5}
+\contentsline {subsection}{\numberline {14.8.6}See Also}{45}{subsection.14.8.6}
+\contentsline {section}{\numberline {14.9}The \textbf {umount.s3ql} command}{46}{section.14.9}
+\contentsline {subsection}{\numberline {14.9.1}Synopsis}{46}{subsection.14.9.1}
+\contentsline {subsection}{\numberline {14.9.2}Description}{46}{subsection.14.9.2}
+\contentsline {subsection}{\numberline {14.9.3}Options}{46}{subsection.14.9.3}
+\contentsline {subsection}{\numberline {14.9.4}Exit Status}{46}{subsection.14.9.4}
+\contentsline {subsection}{\numberline {14.9.5}See Also}{46}{subsection.14.9.5}
+\contentsline {section}{\numberline {14.10}The \textbf {fsck.s3ql} command}{46}{section.14.10}
+\contentsline {subsection}{\numberline {14.10.1}Synopsis}{46}{subsection.14.10.1}
+\contentsline {subsection}{\numberline {14.10.2}Description}{47}{subsection.14.10.2}
+\contentsline {subsection}{\numberline {14.10.3}Options}{47}{subsection.14.10.3}
+\contentsline {subsection}{\numberline {14.10.4}Exit Status}{47}{subsection.14.10.4}
+\contentsline {subsection}{\numberline {14.10.5}See Also}{47}{subsection.14.10.5}
+\contentsline {section}{\numberline {14.11}The \textbf {pcp} command}{47}{section.14.11}
+\contentsline {subsection}{\numberline {14.11.1}Synopsis}{47}{subsection.14.11.1}
+\contentsline {subsection}{\numberline {14.11.2}Description}{48}{subsection.14.11.2}
+\contentsline {subsection}{\numberline {14.11.3}Options}{48}{subsection.14.11.3}
+\contentsline {subsection}{\numberline {14.11.4}Exit Status}{48}{subsection.14.11.4}
+\contentsline {subsection}{\numberline {14.11.5}See Also}{48}{subsection.14.11.5}
+\contentsline {section}{\numberline {14.12}The \textbf {expire\_backups} command}{48}{section.14.12}
+\contentsline {subsection}{\numberline {14.12.1}Synopsis}{48}{subsection.14.12.1}
+\contentsline {subsection}{\numberline {14.12.2}Description}{48}{subsection.14.12.2}
+\contentsline {subsection}{\numberline {14.12.3}Options}{49}{subsection.14.12.3}
+\contentsline {subsection}{\numberline {14.12.4}Exit Status}{49}{subsection.14.12.4}
+\contentsline {subsection}{\numberline {14.12.5}See Also}{49}{subsection.14.12.5}
+\contentsline {chapter}{\numberline {15}Further Resources / Getting Help}{51}{chapter.15}
diff --git a/doc/man/fsck.s3ql.1 b/doc/man/fsck.s3ql.1
index e6c786d..d06ff1e 100644
--- a/doc/man/fsck.s3ql.1
+++ b/doc/man/fsck.s3ql.1
@@ -1,4 +1,4 @@
-.TH "FSCK.S3QL" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "FSCK.S3QL" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
fsck.s3ql \- Check an S3QL file system for errors
.
@@ -45,30 +45,9 @@ which only briefly document the available userspace commands).
.sp
The \fBmkfs.s3ql\fP command checks the new file system in the location
specified by \fIstorage url\fP for errors and attempts to repair any
-problems.
-.sp
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-.SS Amazon S3
-.sp
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \fBs3://<bucketname>\fP. Bucket names must conform to the S3 Bucket
-Name Restrictions.
-.SS Local
-.sp
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\fBlocal://<path>\fP. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \fBlocal:///var/archive\fP.
-.SS SFTP
-.sp
-The storage URL for SFTP connections has the form
-.sp
-.nf
-.ft C
-sftp://<host>[:port]/<path>
-.ft P
-.fi
+problems. The storage url depends on the backend that is used. The
+S3QL User\(aqs Guide should be consulted for a description of the
+available backends.
.SH OPTIONS
.sp
The \fBmkfs.s3ql\fP command accepts the following options.
@@ -76,49 +55,38 @@ The \fBmkfs.s3ql\fP command accepts the following options.
.INDENT 3.5
.INDENT 0.0
.TP
-.BI \-\-homedir \ <path>
-.
-Directory for log files, cache and authentication info.
-(default: \fB~/.s3ql)\fP
+.BI \-\-log \ <target>
+Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify \fBnone\fP to disable logging. Default:
+\fB~/.s3ql/fsck.log\fP
+.TP
+.BI \-\-cachedir \ <path>
+Store cached data in this directory (default: \fB~/.s3ql)\fP
+.TP
+.BI \-\-authfile \ <path>
+Read authentication credentials from this file (default:
+\fB~/.s3ql/authinfo2)\fP
.TP
.BI \-\-debug \ <module>
-.
activate debugging output from <module>. Use \fBall\fP to get
debug messages from all modules. This option can be
specified multiple times.
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.TP
-.B \-\-ssl
-.
-Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext even
-for unencrypted file systems.
-.TP
.B \-\-batch
-.
If user input is required, exit without prompting.
.TP
.B \-\-force
-.
Force checking even if file system is marked clean.
.UNINDENT
.UNINDENT
.UNINDENT
-.SH FILES
-.sp
-Authentication data for backends and bucket encryption passphrases are
-read from \fBauthinfo\fP in \fB~/.s3ql\fP or the directory
-specified with \fB\-\-homedir\fP. Log files are placed in the same
-directory.
.SH EXIT STATUS
.sp
\fBmkfs.s3ql\fP returns exit code 0 if the operation succeeded and 1 if some
diff --git a/doc/man/mkfs.s3ql.1 b/doc/man/mkfs.s3ql.1
index dc57add..9af4449 100644
--- a/doc/man/mkfs.s3ql.1
+++ b/doc/man/mkfs.s3ql.1
@@ -1,4 +1,4 @@
-.TH "MKFS.S3QL" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "MKFS.S3QL" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
mkfs.s3ql \- Create an S3QL file system
.
@@ -44,30 +44,14 @@ sure to consult the full documentation (rather than just the man pages
which only briefly document the available userspace commands).
.sp
The \fBmkfs.s3ql\fP command creates a new file system in the location
-specified by \fIstorage url\fP.
+specified by \fIstorage url\fP. The storage url depends on the backend
+that is used. The S3QL User\(aqs Guide should be consulted for a
+description of the available backends.
.sp
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-.SS Amazon S3
-.sp
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \fBs3://<bucketname>\fP. Bucket names must conform to the S3 Bucket
-Name Restrictions.
-.SS Local
-.sp
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\fBlocal://<path>\fP. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \fBlocal:///var/archive\fP.
-.SS SFTP
-.sp
-The storage URL for SFTP connections has the form
-.sp
-.nf
-.ft C
-sftp://<host>[:port]/<path>
-.ft P
-.fi
+Unless you have specified the \fB\-\-plain\fP option, \fBmkfs.s3ql\fP will ask
+you to enter an encryption password. This password will \fInot\fP be read
+from an authentication file specified with the \fB\-\-authfile\fP
+option to prevent accidental creation of an encrypted bucket.
.SH OPTIONS
.sp
The \fBmkfs.s3ql\fP command accepts the following options.
@@ -75,63 +59,38 @@ The \fBmkfs.s3ql\fP command accepts the following options.
.INDENT 3.5
.INDENT 0.0
.TP
-.BI \-\-homedir \ <path>
-.
-Directory for log files, cache and authentication
-info. (default: \fB~/.s3ql)\fP
+.BI \-\-cachedir \ <path>
+Store cached data in this directory (default: \fB~/.s3ql)\fP
+.TP
+.BI \-\-authfile \ <path>
+Read authentication credentials from this file (default:
+\fB~/.s3ql/authinfo2)\fP
.TP
.BI \-\-debug \ <module>
-.
activate debugging output from <module>. Use \fBall\fP to
-get debug messages from all modules. This option can
-be specified multiple times.
+get debug messages from all modules. This option can be
+specified multiple times.
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.TP
-.B \-\-ssl
-.
-Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
-.TP
-.BI \-\-s3\-location \ <name>
-.
-Storage location for new S3 buckets. Allowed values:
-\fBEU\fP, \fBus\-west\-1\fP, \fBap\-southeast\-1\fP, or \fBus\-standard\fP.
-(default: EU)
-.TP
.BI \-L \ <name>
-.
Filesystem label
.TP
.BI \-\-blocksize \ <size>
-.
Maximum block size in KB (default: 10240)
.TP
.B \-\-plain
-.
Create unencrypted file system.
.TP
.B \-\-force
-.
Overwrite any existing data.
.UNINDENT
.UNINDENT
.UNINDENT
-.SH FILES
-.sp
-Authentication data for backends and bucket encryption passphrases are
-read from \fBauthinfo\fP in \fB~/.s3ql\fP or the directory
-specified with \fB\-\-homedir\fP. Log files are placed in the same
-directory.
.SH EXIT STATUS
.sp
\fBmkfs.s3ql\fP returns exit code 0 if the operation succeeded and 1 if some
diff --git a/doc/man/mount.s3ql.1 b/doc/man/mount.s3ql.1
index fa20a20..0c10701 100644
--- a/doc/man/mount.s3ql.1
+++ b/doc/man/mount.s3ql.1
@@ -1,4 +1,4 @@
-.TH "MOUNT.S3QL" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "MOUNT.S3QL" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
mount.s3ql \- Mount an S3QL file system
.
@@ -44,30 +44,9 @@ sure to consult the full documentation (rather than just the man pages
which only briefly document the available userspace commands).
.sp
The \fBmount.s3ql\fP command mounts the S3QL file system stored in \fIstorage
-url\fP in the directory \fImount point\fP.
-.sp
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-.SS Amazon S3
-.sp
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \fBs3://<bucketname>\fP. Bucket names must conform to the S3 Bucket
-Name Restrictions.
-.SS Local
-.sp
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\fBlocal://<path>\fP. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \fBlocal:///var/archive\fP.
-.SS SFTP
-.sp
-The storage URL for SFTP connections has the form
-.sp
-.nf
-.ft C
-sftp://<host>[:port]/<path>
-.ft P
-.fi
+url\fP in the directory \fImount point\fP. The storage url depends on the
+backend that is used. The S3QL User\(aqs Guide should be consulted for a
+description of the available backends.
.SH OPTIONS
.sp
The \fBmount.s3ql\fP command accepts the following options.
@@ -75,35 +54,32 @@ The \fBmount.s3ql\fP command accepts the following options.
.INDENT 3.5
.INDENT 0.0
.TP
-.BI \-\-homedir \ <path>
-.
-Directory for log files, cache and authentication
-info. (default: \fB~/.s3ql)\fP
+.BI \-\-log \ <target>
+Write logging info into this file. File will be
+rotated when it reaches 1 MB, and at most 5 old log
+files will be kept. Specify \fBnone\fP to disable
+logging. Default: \fB~/.s3ql/mount.log\fP
+.TP
+.BI \-\-cachedir \ <path>
+Store cached data in this directory (default:
+\fB~/.s3ql)\fP
+.TP
+.BI \-\-authfile \ <path>
+Read authentication credentials from this file
+(default: \fB~/.s3ql/authinfo2)\fP
.TP
.BI \-\-debug \ <module>
-.
activate debugging output from <module>. Use \fBall\fP to
get debug messages from all modules. This option can
be specified multiple times.
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.TP
-.B \-\-ssl
-.
-Use SSL when connecting to remote servers. This option
-is not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
-.TP
.BI \-\-cachesize \ <size>
-.
Cache size in kb (default: 102400 (100 MB)). Should be
at least 10 times the blocksize of the filesystem,
otherwise an object may be retrieved and written
@@ -111,7 +87,6 @@ several times during a single write() or read()
operation.
.TP
.BI \-\-max\-cache\-entries \ <num>
-.
Maximum number of entries in cache (default: 768).
Each cache entry requires one file descriptor, so if
you increase this number you have to make sure that
@@ -120,7 +95,6 @@ your process file descriptor limit (as set with
cache entries + 100).
.TP
.B \-\-allow\-other
-.
Normally, only the user who called \fBmount.s3ql\fP can
access the mount point. This user then also has full
access to it, independent of individual file
@@ -130,53 +104,43 @@ well and individual file permissions are taken into
account for all users.
.TP
.B \-\-allow\-root
-.
Like \fB\-\-allow\-other\fP, but restrict access to the
mounting user and the root user.
.TP
.B \-\-fg
-.
Do not daemonize, stay in foreground
.TP
.B \-\-single
-.
Run in single threaded mode. If you don\(aqt understand
this, then you don\(aqt need it.
.TP
.B \-\-upstart
-.
Stay in foreground and raise SIGSTOP once mountpoint
is up.
.TP
.B \-\-profile
-.
Create profiling information. If you don\(aqt understand
this, then you don\(aqt need it.
.TP
.BI \-\-compress \ <name>
-.
Compression algorithm to use when storing new data.
Allowed values: \fBlzma\fP, \fBbzip2\fP, \fBzlib\fP, none.
(default: \fBlzma\fP)
.TP
.BI \-\-metadata\-upload\-interval \ <seconds>
-.
Interval in seconds between complete metadata uploads.
Set to 0 to disable. Default: 24h.
.TP
-.BI \-\-compression\-threads \ <no>
-.
-Number of parallel compression and encryption threads
-to use (default: 1).
+.BI \-\-threads \ <no>
+Number of parallel upload threads to use (default:
+auto).
+.TP
+.B \-\-nfs
+Support export of S3QL file systems over NFS (default:
+False)
.UNINDENT
.UNINDENT
.UNINDENT
-.SH FILES
-.sp
-Authentication data for backends and bucket encryption passphrases are
-read from \fBauthinfo\fP in \fB~/.s3ql\fP or the directory
-specified with \fB\-\-homedir\fP. Log files are placed in the same
-directory.
.SH EXIT STATUS
.sp
\fBmount.s3ql\fP returns exit code 0 if the operation succeeded and 1 if some
diff --git a/doc/man/s3qladm.1 b/doc/man/s3qladm.1
index 24cb81a..805f831 100644
--- a/doc/man/s3qladm.1
+++ b/doc/man/s3qladm.1
@@ -1,4 +1,4 @@
-.TH "S3QLADM" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "S3QLADM" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
s3qladm \- Manage S3QL buckets
.
@@ -50,28 +50,8 @@ The \fBs3qladm\fP command performs various operations on S3QL buckets.
The file system contained in the bucket \fImust not be mounted\fP when
using \fBs3qladm\fP or things will go wrong badly.
.sp
-The form of the storage url depends on the backend that is used. The
-following backends are supported:
-.SS Amazon S3
-.sp
-To store your file system in an Amazon S3 bucket, use a storage URL of
-the form \fBs3://<bucketname>\fP. Bucket names must conform to the S3 Bucket
-Name Restrictions.
-.SS Local
-.sp
-The local backend stores file system data in a directory on your
-computer. The storage URL for the local backend has the form
-\fBlocal://<path>\fP. Note that you have to write three consecutive
-slashes to specify an absolute path, e.g. \fBlocal:///var/archive\fP.
-.SS SFTP
-.sp
-The storage URL for SFTP connections has the form
-.sp
-.nf
-.ft C
-sftp://<host>[:port]/<path>
-.ft P
-.fi
+The storage url depends on the backend that is used. The S3QL User\(aqs
+Guide should be consulted for a description of the available backends.
.SH OPTIONS
.sp
The \fBs3qladm\fP command accepts the following options.
@@ -80,31 +60,28 @@ The \fBs3qladm\fP command accepts the following options.
.INDENT 0.0
.TP
.BI \-\-debug \ <module>
-.
activate debugging output from <module>. Use \fBall\fP to get
debug messages from all modules. This option can be
specified multiple times.
.TP
.B \-\-quiet
-.
be really quiet
.TP
-.BI \-\-homedir \ <path>
-.
-Directory for log files, cache and authentication info.
-(default: \fB~/.s3ql)\fP
+.BI \-\-log \ <target>
+Write logging info into this file. File will be rotated
+when it reaches 1 MB, and at most 5 old log files will be
+kept. Specify \fBnone\fP to disable logging. Default:
+\fBnone\fP
+.TP
+.BI \-\-authfile \ <path>
+Read authentication credentials from this file (default:
+\fB~/.s3ql/authinfo2)\fP
+.TP
+.BI \-\-cachedir \ <path>
+Store cached data in this directory (default: \fB~/.s3ql)\fP
.TP
.B \-\-version
-.
just print program version and exit
-.TP
-.B \-\-ssl
-.
-Use SSL when connecting to remote servers. This option is
-not enabled by default, because for encrypted file
-systems, all data is already encrypted anyway, and
-authentication data is never transmitted in plaintext
-even for unencrypted file systems.
.UNINDENT
.UNINDENT
.UNINDENT
@@ -117,27 +94,17 @@ The following actions may be specified:
.INDENT 0.0
.TP
.B passphrase
-.
Changes the encryption passphrase of the bucket.
.TP
.B upgrade
-.
Upgrade the file system contained in the bucket to the newest revision.
.TP
.B delete
-.
Delete the bucket and all its contents.
.TP
.B download\-metadata
-.
Interactively download backups of the file system metadata.
.UNINDENT
-.SH FILES
-.sp
-Authentication data for backends and bucket encryption passphrases are
-read from \fBauthinfo\fP in \fB~/.s3ql\fP or the directory
-specified with \fB\-\-homedir\fP. Log files are placed in the same
-directory.
.SH EXIT STATUS
.sp
\fBs3qladm\fP returns exit code 0 if the operation succeeded and 1 if some
diff --git a/doc/man/s3qlcp.1 b/doc/man/s3qlcp.1
index b2bc8b2..e73d9a9 100644
--- a/doc/man/s3qlcp.1
+++ b/doc/man/s3qlcp.1
@@ -1,4 +1,4 @@
-.TH "S3QLCP" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "S3QLCP" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
s3qlcp \- Copy-on-write replication on S3QL file systems
.
@@ -58,30 +58,22 @@ the root user. This limitation might be removed in the future (see \fI\%issue 15
Note that:
.INDENT 0.0
.IP \(bu 2
-.
After the replication, both source and target directory will still
be completely ordinary directories. You can regard \fB<src>\fP as a
snapshot of \fB<target>\fP or vice versa. However, the most common
usage of \fBs3qlcp\fP is to regularly duplicate the same source
directory, say \fBdocuments\fP, to different target directories. For a
e.g. monthly replication, the target directories would typically be
-named something like \fBdocuments_Januray\fP for the replication in
+named something like \fBdocuments_January\fP for the replication in
January, \fBdocuments_February\fP for the replication in February etc.
In this case it is clear that the target directories should be
regarded as snapshots of the source directory.
.IP \(bu 2
-.
Exactly the same effect could be achieved by an ordinary copy
program like \fBcp \-a\fP. However, this procedure would be orders of
magnitude slower, because \fBcp\fP would have to read every file
completely (so that S3QL had to fetch all the data over the network
from the backend) before writing them into the destination folder.
-.IP \(bu 2
-.
-Before starting with the replication, S3QL has to flush the local
-cache. So if you just copied lots of new data into the file system
-that has not yet been uploaded, replication will take longer than
-usual.
.UNINDENT
.SS Snapshotting vs Hardlinking
.sp
@@ -93,13 +85,11 @@ identical file already exists in the backup. However, using hardlinks
has two large disadvantages:
.INDENT 0.0
.IP \(bu 2
-.
backups and restores always have to be made with a special program
that takes care of the hardlinking. The backup must not be touched
by any other programs (they may make changes that inadvertently
affect other hardlinked files)
.IP \(bu 2
-.
special care needs to be taken to handle files which are already
hardlinked (the restore program needs to know that the hardlink was
not just introduced by the backup program to safe space)
@@ -115,15 +105,12 @@ The \fBs3qlcp\fP command accepts the following options:
.INDENT 0.0
.TP
.B \-\-debug
-.
activate debugging output
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.UNINDENT
.UNINDENT
diff --git a/doc/man/s3qlctrl.1 b/doc/man/s3qlctrl.1
index f8c0d56..9da182d 100644
--- a/doc/man/s3qlctrl.1
+++ b/doc/man/s3qlctrl.1
@@ -1,4 +1,4 @@
-.TH "S3QLCTRL" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "S3QLCTRL" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
s3qlctrl \- Control a mounted S3QL file system
.
@@ -50,20 +50,22 @@ which only briefly document the available userspace commands).
The \fBs3qlctrl\fP command performs various actions on the S3QL file system mounted
in \fBmountpoint\fP.
.sp
+\fBs3qlctrl\fP can only be called by the user that mounted the file system
+and (if the file system was mounted with \fB\-\-allow\-other\fP or
+\fB\-\-allow\-root\fP) the root user. This limitation might be
+removed in the future (see \fI\%issue 155\fP).
+.sp
The following actions may be specified:
.INDENT 0.0
.TP
.B flushcache
-.
Uploads all changed file data to the backend.
.TP
.B upload\-meta
-.
Upload metadata to the backend. All file system operations will
block while a snapshot of the metadata is prepared for upload.
.TP
.B cachesize
-.
Changes the cache size of the file system. This action requires an
additional argument that specifies the new cache size in kB, so the
complete command line is:
@@ -75,7 +77,6 @@ s3qlctrl [options] cachesize <mountpoint> <new\-cache\-size>
.fi
.TP
.B log
-.
Change the amount of information that is logged into
\fB~/.s3ql/mount.log\fP file. The complete syntax is:
.sp
@@ -99,15 +100,12 @@ what specific action is being invoked:
.INDENT 0.0
.TP
.B \-\-debug
-.
activate debugging output
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.UNINDENT
.UNINDENT
diff --git a/doc/man/s3qllock.1 b/doc/man/s3qllock.1
index 5892a46..4f4f88d 100644
--- a/doc/man/s3qllock.1
+++ b/doc/man/s3qllock.1
@@ -1,4 +1,4 @@
-.TH "S3QLLOCK" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "S3QLLOCK" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
s3qllock \- Make trees on an S3QL file system immutable
.
@@ -48,6 +48,11 @@ system immutable. Immutable trees can no longer be changed in any way
whatsoever. You can not add new files or directories and you can not
change or delete existing files and directories. The only way to get
rid of an immutable tree is to use the \fBs3qlrm\fP command.
+.sp
+\fBs3qllock\fP can only be called by the user that mounted the file system
+and (if the file system was mounted with \fB\-\-allow\-other\fP or
+\fB\-\-allow\-root\fP) the root user. This limitation might be
+removed in the future (see \fI\%issue 155\fP).
.SH RATIONALE
.sp
Immutability is a feature designed for backups. Traditionally, backups
@@ -87,15 +92,12 @@ The \fBs3qllock\fP command accepts the following options:
.INDENT 0.0
.TP
.B \-\-debug
-.
activate debugging output
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.UNINDENT
.UNINDENT
diff --git a/doc/man/s3qlrm.1 b/doc/man/s3qlrm.1
index 66af40f..1a4a09e 100644
--- a/doc/man/s3qlrm.1
+++ b/doc/man/s3qlrm.1
@@ -1,4 +1,4 @@
-.TH "S3QLRM" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "S3QLRM" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
s3qlrm \- Fast tree removal on S3QL file systems
.
@@ -51,6 +51,11 @@ you to delete immutable trees (which can be created with
.sp
Be warned that there is no additional confirmation. The directory will
be removed entirely and immediately.
+.sp
+\fBs3qlrm\fP can only be called by the user that mounted the file system
+and (if the file system was mounted with \fB\-\-allow\-other\fP or
+\fB\-\-allow\-root\fP) the root user. This limitation might be
+removed in the future (see \fI\%issue 155\fP).
.SH OPTIONS
.sp
The \fBs3qlrm\fP command accepts the following options:
@@ -59,15 +64,12 @@ The \fBs3qlrm\fP command accepts the following options:
.INDENT 0.0
.TP
.B \-\-debug
-.
activate debugging output
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.UNINDENT
.UNINDENT
diff --git a/doc/man/s3qlstat.1 b/doc/man/s3qlstat.1
index b9ce4f6..b2849e2 100644
--- a/doc/man/s3qlstat.1
+++ b/doc/man/s3qlstat.1
@@ -1,4 +1,4 @@
-.TH "S3QLSTAT" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "S3QLSTAT" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
s3qlstat \- Gather S3QL file system statistics
.
@@ -58,15 +58,12 @@ The \fBs3qlstat\fP command accepts the following options:
.INDENT 0.0
.TP
.B \-\-debug
-.
activate debugging output
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.UNINDENT
.UNINDENT
diff --git a/doc/man/umount.s3ql.1 b/doc/man/umount.s3ql.1
index b1ea3e8..cfc4bdd 100644
--- a/doc/man/umount.s3ql.1
+++ b/doc/man/umount.s3ql.1
@@ -1,4 +1,4 @@
-.TH "UMOUNT.S3QL" "1" "May 20, 2011" "1.0.1" "S3QL"
+.TH "UMOUNT.S3QL" "1" "September 28, 2011" "1.2" "S3QL"
.SH NAME
umount.s3ql \- Unmount an S3QL file system
.
@@ -63,19 +63,15 @@ The \fBumount.s3ql\fP command accepts the following options.
.INDENT 0.0
.TP
.B \-\-debug
-.
activate debugging output
.TP
.B \-\-quiet
-.
be really quiet
.TP
.B \-\-version
-.
just print program version and exit
.TP
.B \-\-lazy, \-z
-.
Lazy umount. Detaches the file system immediately, even if
there are still open files. The data will be uploaded in the
background once all open files have been closed.
diff --git a/doc/manual.pdf b/doc/manual.pdf
index 0553294..c6558d7 100644
--- a/doc/manual.pdf
+++ b/doc/manual.pdf
Binary files differ