|author||Dimitri John Ledkov <firstname.lastname@example.org>||2018-02-19 15:51:31 +0000|
|committer||Dimitri John Ledkov <email@example.com>||2018-02-19 15:52:49 +0000|
Diffstat (limited to 'tests/README.md')
1 files changed, 61 insertions, 25 deletions
diff --git a/tests/README.md b/tests/README.md
index 04d2ce2a..d4b80da1 100644
@@ -45,46 +45,51 @@ $ make TEST=001\* test-fsck
$ TEST=001\* ./fsck-tests.sh
-will run the first test in fsck-tests subdirectory.
+will run the first test in fsck-tests subdirectory. If the test directories
+follow a good naming scheme, it's possible to select a subset eg. like the
+convert tests for ext filesystems.
## Test structure
- * tests targeted at bugs that are fixable by fsck
+ * tests targeted at bugs that are fixable by fsck, the test directory can
+ contain images that will get fixed, or a custom script `./test.sh` that
+ will be run if present
- * coverage tests of ext2/3/4 and btrfs-convert options
+ * coverage tests of ext2/3/4 or reiserfs and btrfs-convert options
* collection of fuzzed or crafted images
* tests that are supposed to run various utilities on the images and not
* tests for command line interface, option coverage, weird option combinations that should not work
* not necessary to do any functional testing, could be rather lightweight
* functional tests should go to to other test dirs
* the driver script will only execute `./test.sh` in the test directory
* anything that does not fit to the above, the test driver script will only
execute `./test.sh` in the test directory
- * script with shell helpers, separated by functionality
+ * scripts with shell helpers, separated by functionality
- * default testing image, the file is never deleted by the scripts but
- truncated to 0 bytes, so it keeps it's permissions. It's eg. possible to
- host it on NFS, make it `chmod a+w` for root.
+ * default testing image, available as `TEST_DEV` variable, the file is never
+ deleted by the scripts but truncated to 0 bytes, so it keeps it's
+ permissions. It's eg. possible to host it on NFS, make it `chmod a+w` for
## Other tuning, environment variables
@@ -125,10 +130,10 @@ Multiple values can be separated by `,`.
-Some commands require root privileges (to mount/umount, access loop devices).
-It is assumed that `sudo` will work in some way (no password, password asked
-and cached). Note that instrumentation is not applied in this case, for safety
-reasons. You need to modify the test script instead.
+Some commands require root privileges (to mount/umount, access loop devices or
+call privileged ioctls). It is assumed that `sudo` will work in some way (no
+password, password asked and cached). Note that instrumentation is not applied
+in this case, for safety reasons. You need to modify the test script instead.
@@ -143,13 +148,14 @@ the loop devices as they are managed on a per-test basis.
### Prototyping tests, quick tests
There's a script `test-console.sh` that will run shell commands in a loop and
-logs the output with the testing environment set up.
+logs the output with the testing environment set up. It sources the common
+helper scripts so the shell functions are available.
### Runtime dependencies
The tests use some common system utilities like `find`, `rm`, `dd`. Additionally,
specific tests need the following packages installed: `acl`, `attr`,
## New test
@@ -158,13 +164,15 @@ specific tests need the following packages installed: `acl`, `attr`,
an easy start copy an existing `test.sh` script from some test that might be
close to the purpose of your new test. The environment setup includes the
common scripts and/or prepares the test devices. Other scripts contain examples
-how to do mkfs, mount, unmount, check, etc.
+how to do mkfs, mount, unmount, check, loop device management etc.
2. Use the highest unused number in the sequence, write a short descriptive title
and join by dashes `-`. This will become the directory name, eg. `012-subvolume-sync-must-wait`.
3. Write a short description of the bug and how it's tested to the comment at the
-begining of `test.sh`. You don't need to add the file to git yet.
+begining of `test.sh`. You don't need to add the file to git yet. Don't forget
+to make the file executable, otherwise it's not going to be executed by the
4. Write the test commands, comment anything that's not obvious.
@@ -178,10 +186,12 @@ $ TEST=012\* ./misc-tests.sh # from tests/
fixed the bug (or both). Subject line of the shall mention the name of the
new directory for ease of search, eg. `btrfs-progs: tests: add 012-subvolume-sync-must-wait`
+7. A commit that fixes a bug should be applied before the test that verifies
+ the fix. This is to keep the git history bisectable.
### Crafted/fuzzed images
-Images that are create by fuzzing or specially crafted to trigger some error
+Images that are created by fuzzing or specially crafted to trigger some error
conditions should be added to the directory *fuzz-tests/images*, accompanied by
a textual description of the source (bugzilla, mail), the reporter, brief
description of the problem or the stack trace.
@@ -191,6 +201,31 @@ the fuzz tests always succeed when run on random checked out. This helps
+# Exported testsuite
+The tests are typically run from git on binaries built from the git sources. It
+is possible to extract only the testsuite files and run it independently. Use
+$ make testsuite
+This will gather scripts and generate `tests/btrfs-progs-tests.tar.gz`. The
+files inside the tar are in the top level directory, make sure you extract
+the contents to an empty directory. From there you can start the tests as
+described above (the non-make variant).
+By default the binaries found in `$PATH` are used, this will normally mean the
+system binaries. You can also override the `$TOP` shell variable and this
+path will be used as prefix for all btrfs binaries inside the tests.
+There are some utilities that are not distributed but are necessary for the
+tests. They are in the top level directory of the testsuite and their path
+cannot be set.
+The tests assume write acesss to their directories.
# Coding style, best practices
@@ -206,8 +241,9 @@ bisectability.
always built when the tests are started through make
* use functions instead of repeating code
* generic helpers could be factored to the `common` script
-* cleanup after successful test
-* use common helpers and variables
+* cleanup files an intermediate state (mount, loop devices, device mapper
+ devices) a after successful test
+* use common helpers and variables where possible
## do not