diff options
Diffstat (limited to 'tests')
104 files changed, 3214 insertions, 0 deletions
diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 00000000..6bb3de49 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,161 @@ +# Btrfs-progs tests + +A testsuite covering functionality of btrfs-progs, ie. the checker, image, mkfs +and similar tools. There are no special requirements on kernel features, the +tests build on top of the core functionality like snapshots and device +management. In some cases optional features are turned on by mkfs and the +filesystem image could be mounted, such tests might fail if there's lack of +support. + +## Quick start + +Run the tests from the top directory: + +```shell +$ make test +$ make test-fsck +$ make test-convert +``` + +or selectively from the `tests/` directory: + +```shell +$ ./fsck-tests.sh +$ ./misc-tests.sh +``` + +The verbose output of the tests is logged into a file named after the test +category, eg. `fsck-tests-results.txt`. + +## Selective testing + +The test are prefixed by a number for ordering and uniqueness. To run a +particular test use: + +```shell +$ make TEST=MASK test +``` + +where `MASK` is a glob expression that will execute only tests +that match the MASK. Here the test number comes handy: + +```shell +$ make TEST=001\* test-fsck +$ TEST=001\* ./fsck-tests.sh +``` + +will run the first test in fsck-tests subdirectory. + + +## Test structure + +*tests/fsck-tests/:* + + * tests targeted at bugs that are fixable by fsck + +*tests/convert-tests/:* + + * coverage tests of ext2/3/4 and btrfs-convert options + +*tests/fuzz-tests/:* + + * collection of fuzzed or crafted images + * tests that are supposed to run various utilities on the images and not + crash + +*tests/cli-tests/:* + + * tests for command line interface, option coverage, weird option combinations that should not work + * not necessary to do any functional testing, could be rather lightweight + * functional tests should go to to other test dirs + * the driver script will only execute `./test.sh` in the test directory + +*tests/misc-tests/:* + + * anything that does not fit to the above, the test driver script will only + execute `./test.sh` in the test directory + +*tests/common:* +*tests/common.convert:* + + * script with shell helpers, separated by functionality + +*tests/test.img:* + + * default testing image, the file is never deleted by the scripts but + truncated to 0 bytes, so it keeps it's permissions. It's eg. possible to + host it on NFS, make it `chmod a+w` for root. + + +## Other tuning, environment variables + +### Instrumentation + +It's possible to wrap the tested commands to utilities that might do more +checking or catch failures at runtime. This can be done by setting the +`INSTRUMENT` environment variable: + +```shell +INSTRUMENT=valgrind ./fuzz-tests.sh # in tests/ +make INSTRUMENT=valgrind test-fuzz # in the top directory +``` + +The variable is prepended to the command *unquoted*, all sorts of shell tricks +are possible. + +Note: instrumentation is not applied to privileged commands (anything that uses +the root helper). + +### Verbosity + +Setting the variable `TEST_LOG=tty` will print all commands executed by some of +the wrappers (`run_check` etc), other commands are not printed to the terminal +(but the full output is in the log). + +### Permissions + +Some commands require root privileges (to mount/umount, access loop devices). +It is assumed that `sudo` will work in some way (no password, password asked +and cached). Note that instrumentation is not applied in this case, for safety +reasons. You need to modify the test script instead. + +### Cleanup + +The tests are supposed to cleanup after themselves if they pass. In case of +failure, the rest of the tests are skipped and intermediate files, mounts and +loop devices are kept. This should help to investigate the test failure but at +least the mounts and loop devices need to be cleaned before the next run. + +This is partially done by the script `clean-tests.sh`, you may want to check +the loop devices as they are managed on a per-test basis. + +### Prototyping tests, quick tests + +There's a script `test-console.sh` that will run shell commands in a loop and +logs the output with the testing environment set up. + +## New test + +1. Pick the category for the new test or fallback to `misc-tests` if not sure. For +an easy start copy an existing `test.sh` script from some test that might be +close to the purpose of your new test. The environment setup includes the +common scripts and/or prepares the test devices. Other scripts contain examples +how to do mkfs, mount, unmount, check, etc. + +2. Use the highest unused number in the sequence, write a short descriptive title +and join by dashes `-`. This will become the directory name, eg. `012-subvolume-sync-must-wait`. + +3. Write a short description of the bug and how it's tested to the comment at the +begining of `test.sh`. You don't need to add the file to git yet. + +4. Write the test commands, comment anything that's not obvious. + +5. Test your test. Use the `TEST` variable to jump right to your test: +```shell +$ make TEST=012\* tests-misc # from top directory +$ TEST=012\* ./misc-tests.sh # from tests/ +``` + +6. The commit changelog should reference a commit that either introduced or + fixed the bug (or both). Subject line of the shall mention the name of the + new directory for ease of search, eg. `btrfs-progs: tests: add 012-subvolume-sync-must-wait` diff --git a/tests/clean-tests.sh b/tests/clean-tests.sh new file mode 100755 index 00000000..f7fefdda --- /dev/null +++ b/tests/clean-tests.sh @@ -0,0 +1,24 @@ +#!/bin/sh +# remove all intermediate files from tests + +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +source $TOP/tests/common + +setup_root_helper + +if [ "$BUILD_VERBOSE" = 1 ]; then + verbose=-print +fi + +$SUDO_HELPER umount "$TEST_MNT" &>/dev/null + +if ! cd $TOP/tests; then + echo "ERROR: cannot cd to $TOP/tests" + exit 1 +fi + +find fsck-tests -type f -name '*.restored' $verbose -delete + +# do not remove, the file could have special permissions set +echo -n > test.img diff --git a/tests/cli-tests.sh b/tests/cli-tests.sh new file mode 100755 index 00000000..72f7865a --- /dev/null +++ b/tests/cli-tests.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# +# command line interface coverage tests + +LANG=C +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +TEST_DEV=${TEST_DEV:-} +RESULTS="$TOP/tests/cli-tests-results.txt" +IMAGE="$TOP/tests/test.img" + +source $TOP/tests/common + +export TOP +export RESULTS +export LANG +export IMAGE +export TEST_DEV + +rm -f $RESULTS + +check_prereq btrfs + +# The tests are driven by their custom script called 'test.sh' + +for i in $(find $TOP/tests/cli-tests -maxdepth 1 -mindepth 1 -type d \ + ${TEST:+-name "$TEST"} | sort) +do + name=$(basename $i) + cd $i + if [ -x test.sh ]; then + echo "=== Entering $i" >> $RESULTS + echo " [TEST/cli] $name" + ./test.sh + if [ $? -ne 0 ]; then + _fail "test failed for case $(basename $i)" + fi + fi + cd $TOP +done diff --git a/tests/cli-tests/001-btrfs/test.sh b/tests/cli-tests/001-btrfs/test.sh new file mode 100755 index 00000000..1de2f6f2 --- /dev/null +++ b/tests/cli-tests/001-btrfs/test.sh @@ -0,0 +1,15 @@ +#!/bin/bash +# test commands of btrfs + +source $TOP/tests/common + +check_prereq btrfs + +# returns 1 +run_mayfail $TOP/btrfs || true +run_check $TOP/btrfs version +run_check $TOP/btrfs version -- +run_check $TOP/btrfs help +run_check $TOP/btrfs help -- +run_check $TOP/btrfs --help +run_check $TOP/btrfs --help --full diff --git a/tests/cli-tests/002-balance-full-no-filters/test.sh b/tests/cli-tests/002-balance-full-no-filters/test.sh new file mode 100755 index 00000000..c2757f24 --- /dev/null +++ b/tests/cli-tests/002-balance-full-no-filters/test.sh @@ -0,0 +1,21 @@ +#!/bin/bash +# +# coverage of balance --full-balance + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper +prepare_test_dev 2g + +run_check $TOP/mkfs.btrfs -f $IMAGE +run_check_mount_test_dev + +run_check $SUDO_HELPER $TOP/btrfs balance start --full-balance $TEST_MNT +run_check $SUDO_HELPER $TOP/btrfs balance start $TEST_MNT +run_check $SUDO_HELPER $TOP/btrfs balance --full-balance $TEST_MNT +run_check $SUDO_HELPER $TOP/btrfs balance $TEST_MNT + +run_check_umount_test_dev diff --git a/tests/cli-tests/003-fi-resize-args/test.sh b/tests/cli-tests/003-fi-resize-args/test.sh new file mode 100755 index 00000000..2f136fa2 --- /dev/null +++ b/tests/cli-tests/003-fi-resize-args/test.sh @@ -0,0 +1,46 @@ +#!/bin/bash +# +# test parsing of various resize arguments + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper +prepare_test_dev 2g + +run_check $TOP/mkfs.btrfs -f $IMAGE +run_check_mount_test_dev + +# missing the one of the required arguments +for sep in '' '--'; do + run_check_stdout $TOP/btrfs filesystem resize $sep | + grep -q "btrfs filesystem resize: too few arguments" + run_check_stdout $TOP/btrfs filesystem resize $sep $TEST_MNT | + grep -q "btrfs filesystem resize: too few arguments" + run_check_stdout $TOP/btrfs filesystem resize $sep -128M | + grep -q "btrfs filesystem resize: too few arguments" + run_check_stdout $TOP/btrfs filesystem resize $sep +128M | + grep -q "btrfs filesystem resize: too few arguments" + run_check_stdout $TOP/btrfs filesystem resize $sep 512M | + grep -q "btrfs filesystem resize: too few arguments" + run_check_stdout $TOP/btrfs filesystem resize $sep 1:-128M | + grep -q "btrfs filesystem resize: too few arguments" + run_check_stdout $TOP/btrfs filesystem resize $sep 1:512M | + grep -q "btrfs filesystem resize: too few arguments" + run_check_stdout $TOP/btrfs filesystem resize $sep 1:+128M | + grep -q "btrfs filesystem resize: too few arguments" +done + +# valid resize +for sep in '' '--'; do + run_check $SUDO_HELPER $TOP/btrfs filesystem resize $sep -128M $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs filesystem resize $sep +128M $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs filesystem resize $sep 512M $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs filesystem resize $sep 1:-128M $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs filesystem resize $sep 1:512M $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs filesystem resize $sep 1:+128M $TEST_MNT +done + +run_check_umount_test_dev diff --git a/tests/common b/tests/common new file mode 100644 index 00000000..c50b661f --- /dev/null +++ b/tests/common @@ -0,0 +1,274 @@ +#!/bin/bash +# +# Common routines for all tests +# + +_fail() +{ + echo "$*" | tee -a $RESULTS + exit 1 +} + +# log a message to the results file +_log() +{ + echo "$*" | tee -a $RESULTS +} + +_not_run() +{ + echo " [NOTRUN] $*" + exit 0 +} + +run_check() +{ + echo "############### $@" >> $RESULTS 2>&1 + if [ "$TEST_LOG" = 'tty' ]; then echo "CMD: $@" > /dev/tty; fi + if [ "$1" = 'root_helper' ]; then + "$@" >> $RESULTS 2>&1 || _fail "failed: $@" + else + $INSTRUMENT "$@" >> $RESULTS 2>&1 || _fail "failed: $@" + fi +} + +# same as run_check but the stderr+stdout output is duplicated on stdout and +# can be processed further +run_check_stdout() +{ + echo "############### $@" >> $RESULTS 2>&1 + if [ "$TEST_LOG" = 'tty' ]; then echo "CMD(stdout): $@" > /dev/tty; fi + if [ "$1" = 'root_helper' ]; then + "$@" 2>&1 | tee -a $RESULTS || _fail "failed: $@" + else + $INSTRUMENT "$@" 2>&1 | tee -a $RESULTS || _fail "failed: $@" + fi +} + +# same as run_check but does not fail the test, output is logged +run_mayfail() +{ + echo "############### $@" >> $RESULTS 2>&1 + if [ "$TEST_LOG" = 'tty' ]; then echo "CMD(mayfail): $@" > /dev/tty; fi + if [ "$1" = 'root_helper' ]; then + "$@" >> $RESULTS 2>&1 + else + $INSTRUMENT "$@" >> $RESULTS 2>&1 + fi + if [ $? != 0 ]; then + echo "failed (ignored): $@" >> $RESULTS + return 1 + fi +} + +# first argument is error message to print if it fails, otherwise +# same as run_check but expects the command to fail, output is logged +run_mustfail() +{ + local msg + + msg="$1" + shift + + echo "############### $@" >> $RESULTS 2>&1 + if [ "$TEST_LOG" = 'tty' ]; then echo "CMD(mustfail): $@" > /dev/tty; fi + if [ "$1" = 'root_helper' ]; then + "$@" >> $RESULTS 2>&1 + else + $INSTRUMENT "$@" >> $RESULTS 2>&1 + fi + if [ $? != 0 ]; then + echo "failed (expected): $@" >> $RESULTS + return 0 + else + echo "succeeded (unexpected!): $@" >> $RESULTS + _fail "unexpected success: $msg" + return 1 + fi +} + +check_prereq() +{ + if ! [ -f $TOP/$1 ]; then + _fail "Failed prerequisites: $1"; + fi +} + +check_global_prereq() +{ + which $1 &> /dev/null + if [ $? -ne 0 ]; then + _fail "Failed system wide prerequisities: $1"; + fi +} + +check_image() +{ + local image + + image=$1 + echo "testing image $(basename $image)" >> $RESULTS + $TOP/btrfs check $image >> $RESULTS 2>&1 + [ $? -eq 0 ] && _fail "btrfs check should have detected corruption" + + run_check $TOP/btrfs check --repair $image + run_check $TOP/btrfs check $image +} + +# Extract a usable image from packed formats +# - raw btrfs filesystem images, suffix .raw +# - dtto compressed by XZ, suffix .raw.xz +# - meta-dump images with suffix .img +# - dtto compressed by XZ, suffix .img.xz +extract_image() +{ + local image + local cleanme + + image="$1" + case "$image" in + *.img) + rm -f $image.restored + : ;; + *.img.xz) + xz --decompress --keep "$image" || \ + _fail "failed to decompress image $image" >&2 + image=${image%%.xz} + rm -f $image.restored + cleanme=$image + ;; + *.raw) + cp --sparse=auto $image $image.restored + ;; + *.raw.xz) + xz --decompress --keep "$image" || \ + _fail "failed to decompress image $image" >&2 + image=${image%%.xz} + mv "$image" "$image".restored + ;; + esac + + if ! [ -f $image.restored ]; then + echo "restoring image $(basename $image)" >> $RESULTS + $TOP/btrfs-image -r $image $image.restored \ + &>> $RESULTS \ + || _fail "failed to restore image $image" >&2 + fi + + [ -f "$cleanme" ] && rm -f "$cleanme" + + echo "$image.restored" +} + +# Process all image dumps in a given directory +check_all_images() +{ + local dir + local extracted + + dir="$1" + for image in $(find $dir \( -iname '*.img' -o \ + -iname '*.img.xz' -o \ + -iname '*.raw' -o \ + -iname '*.raw.xz' \) | sort) + do + extracted=$(extract_image "$image") + check_image "$extracted" + rm -f "$extracted" + done +} + +# some tests need to mount the recovered image and do verifications call +# 'setup_root_helper' and then check for have_root_helper == 1 if the test +# needs to fail otherwise; using sudo by default for now +SUDO_HELPER= +NEED_SUDO_VALIDATE=unknown +export SUDO_HELPER +export NEED_SUDO_VALIDATE +root_helper() +{ + if [ $UID -eq 0 ]; then + "$@" + else + if [ "$NEED_SUDO_VALIDATE" = 'yes' ]; then + sudo -v -n &>/dev/null || \ + _not_run "Need to validate sudo credentials" + sudo -n "$@" + elif [ "$NEED_SUDO_VALIDATE" = 'no' ]; then + sudo -n /bin/true &> /dev/null || \ + _not_run "Need to validate sudo user settings" + sudo -n "$@" + else + # should not happen + _not_run "Need to validate root privileges" + fi + fi +} + +setup_root_helper() +{ + if [ $UID -eq 0 -o -n "$SUDO_HELPER" ]; then + return + fi + + # Test for old sudo or special settings, which make sudo -v fail even + # if user setting is NOPASSWD + sudo -n /bin/true &>/dev/null && NEED_SUDO_VALIDATE=no + + # Newer sudo or default sudo setting + sudo -v -n &>/dev/null && NEED_SUDO_VALIDATE=yes + + if [ "$NEED_SUDO_VALIDATE" = 'unknown' ]; then + _not_run "Need to validate root privileges" + fi + SUDO_HELPER=root_helper +} + +prepare_test_dev() +{ + # num[K/M/G/T...] + local size="$1" + + [[ "$TEST_DEV" ]] && return + [[ "$size" ]] || size='2G' + + echo "\$TEST_DEV not given, use $TOP/test/test.img as fallback" >> \ + $RESULTS + TEST_DEV="$TOP/tests/test.img" + + truncate -s "$size" "$TEST_DEV" || _not_run "create file for loop device failed" +} + +run_check_mount_test_dev() +{ + setup_root_helper + + local loop_opt + if [[ -b "$TEST_DEV" ]]; then + loop_opt="" + elif [[ -f "$TEST_DEV" ]]; then + loop_opt="-o loop" + else + _fail "Invalid \$TEST_DEV: $TEST_DEV" + fi + + [[ -d "$TEST_MNT" ]] || { + _fail "Invalid \$TEST_MNT: $TEST_MNT" + } + + run_check $SUDO_HELPER mount $loop_opt "$@" "$TEST_DEV" "$TEST_MNT" +} + +run_check_umount_test_dev() +{ + setup_root_helper + run_check $SUDO_HELPER umount "$@" "$TEST_DEV" +} + +init_env() +{ + TEST_MNT="${TEST_MNT:-$TOP/tests/mnt}" + export TEST_MNT + mkdir -p "$TEST_MNT" || { echo "Failed mkdir -p $TEST_MNT"; exit 1; } +} +init_env diff --git a/tests/common.convert b/tests/common.convert new file mode 100644 index 00000000..4e3d49c9 --- /dev/null +++ b/tests/common.convert @@ -0,0 +1,182 @@ +#!/bin/bash +# helpers for btrfs-convert tests + +# how many files to create. +DATASET_SIZE=50 + +generate_dataset() { + + dataset_type="$1" + dirpath=$TEST_MNT/$dataset_type + run_check $SUDO_HELPER mkdir -p $dirpath + + case $dataset_type in + small) + for num in $(seq 1 $DATASET_SIZE); do + run_check $SUDO_HELPER dd if=/dev/urandom of=$dirpath/$dataset_type.$num bs=10K \ + count=1 >/dev/null 2>&1 + done + ;; + + hardlink) + for num in $(seq 1 $DATASET_SIZE); do + run_check $SUDO_HELPER touch $dirpath/$dataset_type.$num + run_check $SUDO_HELPER ln $dirpath/$dataset_type.$num $dirpath/hlink.$num + done + ;; + + symlink) + for num in $(seq 1 $DATASET_SIZE); do + run_check $SUDO_HELPER touch $dirpath/$dataset_type.$num + run_check $SUDO_HELPER ln -s $dirpath/$dataset_type.$num $dirpath/slink.$num + done + ;; + + brokenlink) + for num in $(seq 1 $DATASET_SIZE); do + run_check $SUDO_HELPER ln -s $dirpath/$dataset_type.$num $dirpath/blink.$num + done + ;; + + perm) + for modes in 777 775 755 750 700 666 664 644 640 600 444 440 400 000 \ + 1777 1775 1755 1750 1700 1666 1664 1644 1640 1600 1444 1440 1400 1000 \ + 2777 2775 2755 2750 2700 2666 2664 2644 2640 2600 2444 2440 2400 2000 \ + 4777 4775 4755 4750 4700 4666 4664 4644 4640 4600 4444 4440 4400 4000; do + if [[ "$modes" == *9* ]] || [[ "$modes" == *8* ]] + then + continue; + else + run_check $SUDO_HELPER touch $dirpath/$dataset_type.$modes + run_check $SUDO_HELPER chmod $modes $dirpath/$dataset_type.$modes + fi + done + ;; + + sparse) + for num in $(seq 1 $DATASET_SIZE); do + run_check $SUDO_HELPER dd if=/dev/urandom of=$dirpath/$dataset_type.$num bs=10K \ + count=1 >/dev/null 2>&1 + run_check $SUDO_HELPER truncate -s 500K $dirpath/$dataset_type.$num + run_check $SUDO_HELPER dd if=/dev/urandom of=$dirpath/$dataset_type.$num bs=10K \ + oflag=append conv=notrunc count=1 >/dev/null 2>&1 + run_check $SUDO_HELPER truncate -s 800K $dirpath/$dataset_type.$num + done + ;; + + acls) + for num in $(seq 1 $DATASET_SIZE); do + run_check $SUDO_HELPER touch $dirpath/$dataset_type.$num + run_check $SUDO_HELPER setfacl -m "u:root:x" $dirpath/$dataset_type.$num + run_check $SUDO_HELPER setfattr -n user.foo -v bar$num $dirpath/$dataset_type.$num + done + ;; + esac +} + +populate_fs() { + + for dataset_type in 'small' 'hardlink' 'symlink' 'brokenlink' 'perm' 'sparse' 'acls'; do + generate_dataset "$dataset_type" + done +} + +# verbose message before the test, same arguments as convert_test +convert_test_preamble() { + local features + local msg + + features="$1" + msg="$2" + shift 3 + echo " [TEST/conv] $msg, btrfs" "${features:-defaults}" + echo "creating ext image with: $@" >> $RESULTS +} + +# prepare TEST_DEV before conversion, create filesystem and mount it, image +# size is 512MB +# $@: free form, command to create the filesystem, with appended -F +convert_test_prep_fs() { + # TEST_DEV not removed as the file might have special permissions, eg. + # when test image is on NFS and would not be writable for root + run_check truncate -s 0 $TEST_DEV + # 256MB is the smallest acceptable btrfs image. + run_check truncate -s 512M $TEST_DEV + run_check "$@" -F $TEST_DEV + + # create a file to check btrfs-convert can convert regular file correct + run_check_mount_test_dev + + # create a file inside the fs before convert, to make sure there is + # data covering btrfs backup superblock range (64M) + run_check $SUDO_HELPER dd if=/dev/zero bs=1M count=64 \ + of=$TEST_MNT/convert_space_holder +} + +# generate md5 checksums of files on $TEST_MNT +# $1: path where the checksums will be stored +convert_test_gen_checksums() { + local CHECKSUMTMP + CHECKSUMTMP="$1" + + run_check $SUDO_HELPER dd if=/dev/zero of=$TEST_MNT/test bs=$nodesize \ + count=1 >/dev/null 2>&1 + run_check_stdout $SUDO_HELPER find $TEST_MNT -type f ! -name 'image' -exec md5sum {} \+ > "$CHECKSUMTMP" +} + +# do conversion with given features and nodesize, fsck afterwards +# $1: features, argument of -O, can be empty +# $2: nodesize, argument of -N, can be empty +convert_test_do_convert() { + run_check $TOP/btrfs-convert ${1:+-O "$1"} ${2:+-N "$2"} $TEST_DEV + run_check $TOP/btrfs check $TEST_DEV + run_check $TOP/btrfs-show-super -Ffa $TEST_DEV +} + +# post conversion checks, verify md5sums +# $1: file with checksums +convert_test_post_check() { + local CHECKSUMTMP + CHECKSUMTMP="$1" + + run_check_mount_test_dev + run_check_stdout $SUDO_HELPER md5sum -c "$CHECKSUMTMP" | + grep -q 'FAILED' && _fail "file validation failed" + run_check_umount_test_dev +} + +# do rollback and fsck +convert_test_post_rollback() { + run_check $TOP/btrfs-convert --rollback $TEST_DEV + run_check fsck -n -t ext2,ext3,ext4 $TEST_DEV +} + +# simple wrapper for a convert test +# $1: btrfs features, argument to -O +# $2: description of the test "ext2 8k nodesize" +# $3: nodesize value +# $4 + rest: command to create the ext2 image +convert_test() { + local features + local nodesize + local msg + local CHECKSUMTMP + + features="$1" + msg="$2" + nodesize="$3" + shift 3 + convert_test_preamble "$features" "$msg" "$nodesize" "$@" + convert_test_prep_fs "$@" + populate_fs + CHECKSUMTMP=$(mktemp --tmpdir btrfs-progs-convert.XXXXXXXXXX) + convert_test_gen_checksums "$CHECKSUMTMP" + + run_check_umount_test_dev + + convert_test_do_convert "$features" "$nodesize" + convert_test_post_check "$CHECKSUMTMP" + rm $CHECKSUMTMP + + convert_test_post_rollback +} diff --git a/tests/convert-tests.sh b/tests/convert-tests.sh new file mode 100755 index 00000000..0e025f99 --- /dev/null +++ b/tests/convert-tests.sh @@ -0,0 +1,49 @@ +#!/bin/bash +# +# convert ext2/3/4 images to btrfs images, and make sure the results are +# clean. + +LANG=C +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +TEST_DEV=${TEST_DEV:-} +RESULTS="$TOP/tests/convert-tests-results.txt" +IMAGE="$TOP/tests/test.img" + +source $TOP/tests/common +source $TOP/tests/common.convert + +export TOP +export RESULTS +export LANG +export IMAGE +export TEST_DEV + +rm -f $RESULTS + +run_one_test() { + local testdir + local testname + + testdir="$1" + testname=$(basename "$testdir") + echo " [TEST/conv] $testname" + cd "$testdir" + echo "=== Entering $testname" >> $RESULTS + if [ -x test.sh ]; then + # Only support custom test scripts + ./test.sh + if [ $? -ne 0 ]; then + _fail "test failed for case $testname" + fi + else + _fail "custom test script not found" + fi +} + +# Test special images +for i in $(find $TOP/tests/convert-tests -maxdepth 1 -mindepth 1 -type d \ + ${TEST:+-name "$TEST"} | sort) +do + run_one_test "$i" +done diff --git a/tests/convert-tests/001-ext2-basic/test.sh b/tests/convert-tests/001-ext2-basic/test.sh new file mode 100755 index 00000000..8f4f935d --- /dev/null +++ b/tests/convert-tests/001-ext2-basic/test.sh @@ -0,0 +1,16 @@ +#!/bin/bash + +source $TOP/tests/common +source $TOP/tests/common.convert + +setup_root_helper +prepare_test_dev 512M +check_prereq btrfs-convert + +for feature in '' 'extref' 'skinny-metadata' 'no-holes'; do + convert_test "$feature" "ext2 4k nodesize" 4096 mke2fs -b 4096 + convert_test "$feature" "ext2 8k nodesize" 8192 mke2fs -b 4096 + convert_test "$feature" "ext2 16k nodesize" 16384 mke2fs -b 4096 + convert_test "$feature" "ext2 32k nodesize" 32768 mke2fs -b 4096 + convert_test "$feature" "ext2 64k nodesize" 65536 mke2fs -b 4096 +done diff --git a/tests/convert-tests/002-ext3-basic/test.sh b/tests/convert-tests/002-ext3-basic/test.sh new file mode 100755 index 00000000..aeb111eb --- /dev/null +++ b/tests/convert-tests/002-ext3-basic/test.sh @@ -0,0 +1,16 @@ +#!/bin/bash + +source $TOP/tests/common +source $TOP/tests/common.convert + +setup_root_helper +prepare_test_dev 512M +check_prereq btrfs-convert + +for feature in '' 'extref' 'skinny-metadata' 'no-holes'; do + convert_test "$feature" "ext3 4k nodesize" 4096 mke2fs -j -b 4096 + convert_test "$feature" "ext3 8k nodesize" 8192 mke2fs -j -b 4096 + convert_test "$feature" "ext3 16k nodesize" 16384 mke2fs -j -b 4096 + convert_test "$feature" "ext3 32k nodesize" 32768 mke2fs -j -b 4096 + convert_test "$feature" "ext3 64k nodesize" 65536 mke2fs -j -b 4096 +done diff --git a/tests/convert-tests/003-ext4-basic/test.sh b/tests/convert-tests/003-ext4-basic/test.sh new file mode 100755 index 00000000..531c81bd --- /dev/null +++ b/tests/convert-tests/003-ext4-basic/test.sh @@ -0,0 +1,16 @@ +#!/bin/bash + +source $TOP/tests/common +source $TOP/tests/common.convert + +setup_root_helper +prepare_test_dev 512M +check_prereq btrfs-convert + +for feature in '' 'extref' 'skinny-metadata' 'no-holes'; do + convert_test "$feature" "ext4 4k nodesize" 4096 mke2fs -t ext4 -b 4096 + convert_test "$feature" "ext4 8k nodesize" 8192 mke2fs -t ext4 -b 4096 + convert_test "$feature" "ext4 16k nodesize" 16384 mke2fs -t ext4 -b 4096 + convert_test "$feature" "ext4 32k nodesize" 32768 mke2fs -t ext4 -b 4096 + convert_test "$feature" "ext4 64k nodesize" 65536 mke2fs -t ext4 -b 4096 +done diff --git a/tests/convert-tests/004-ext2-backup-superblock-ranges/drdhdhdrd.e2image.raw.xz b/tests/convert-tests/004-ext2-backup-superblock-ranges/drdhdhdrd.e2image.raw.xz Binary files differnew file mode 100644 index 00000000..73e2309c --- /dev/null +++ b/tests/convert-tests/004-ext2-backup-superblock-ranges/drdhdhdrd.e2image.raw.xz diff --git a/tests/convert-tests/004-ext2-backup-superblock-ranges/drdhdhrh.e2image.raw.xz b/tests/convert-tests/004-ext2-backup-superblock-ranges/drdhdhrh.e2image.raw.xz Binary files differnew file mode 100644 index 00000000..0d5442ff --- /dev/null +++ b/tests/convert-tests/004-ext2-backup-superblock-ranges/drdhdhrh.e2image.raw.xz diff --git a/tests/convert-tests/004-ext2-backup-superblock-ranges/hrhdhdrd.e2image.raw.xz b/tests/convert-tests/004-ext2-backup-superblock-ranges/hrhdhdrd.e2image.raw.xz Binary files differnew file mode 100644 index 00000000..a1a30429 --- /dev/null +++ b/tests/convert-tests/004-ext2-backup-superblock-ranges/hrhdhdrd.e2image.raw.xz diff --git a/tests/convert-tests/004-ext2-backup-superblock-ranges/hrhdhrh.e2image.raw.xz b/tests/convert-tests/004-ext2-backup-superblock-ranges/hrhdhrh.e2image.raw.xz Binary files differnew file mode 100644 index 00000000..ac0dbc60 --- /dev/null +++ b/tests/convert-tests/004-ext2-backup-superblock-ranges/hrhdhrh.e2image.raw.xz diff --git a/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh b/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh new file mode 100755 index 00000000..d85e4de4 --- /dev/null +++ b/tests/convert-tests/004-ext2-backup-superblock-ranges/test.sh @@ -0,0 +1,43 @@ +#!/bin/bash + +# Specially created e2image dump to test backup superblock migration for +# new convert. +# These images will cause the following problems if convert doesn't handle +# backup superblock migration well: +# 1) Assert while building free space tree +# 2) Error copying inodes +# 3) Discontinuous file extents after convert +# 4) Overlap file extents +# 5) Unable to rollback + +source $TOP/tests/common + +check_prereq btrfs-convert +check_prereq btrfs +check_prereq btrfs-show-super +check_global_prereq e2fsck +check_global_prereq xzcat + +setup_root_helper +prepare_test_dev 512M + +# override common function +function check_image() { + TEST_DEV="$1" + run_check e2fsck -n -f $TEST_DEV + run_check $TOP/btrfs-convert $TEST_DEV + run_check $TOP/btrfs check $TEST_DEV + run_check $TOP/btrfs-show-super $TEST_DEV + + run_check_mount_test_dev + run_check $SUDO_HELPER e2fsck -n -f $TEST_MNT/ext2_saved/image + run_check $SUDO_HELPER umount $TEST_MNT + + run_check $TOP/btrfs check $TEST_DEV + run_check $TOP/btrfs-convert -r $TEST_DEV + run_check e2fsck -n -f $TEST_DEV + + rm -f $TEST_DEV +} + +check_all_images diff --git a/tests/convert-tests/005-delete-all-rollback/test.sh b/tests/convert-tests/005-delete-all-rollback/test.sh new file mode 100755 index 00000000..d498e5f8 --- /dev/null +++ b/tests/convert-tests/005-delete-all-rollback/test.sh @@ -0,0 +1,61 @@ +#!/bin/bash +# create a base image, convert to btrfs, remove all files, rollback the ext4 image +# note: ext4 only + +source $TOP/tests/common +source $TOP/tests/common.convert + +setup_root_helper +prepare_test_dev 512M +check_prereq btrfs-convert + +# simple wrapper for a convert test +# $1: btrfs features, argument to -O +# $2: message +# $3: nodesize value +# $4 + rest: command to create the ext2 image +do_test() { + local features + local msg + local nodesize + local CHECKSUMTMP + local here + + features="$1" + msg="$2" + nodesize="$3" + shift 3 + convert_test_preamble "$features" "$msg" "$nodesize" "$@" + convert_test_prep_fs "$@" + populate_fs + CHECKSUMTMP=$(mktemp --tmpdir btrfs-progs-convert.XXXXXXXXXX) + convert_test_gen_checksums "$CHECKSUMTMP" + + run_check_umount_test_dev + + convert_test_do_convert "$features" "$nodesize" + convert_test_post_check "$CHECKSUMTMP" + + run_check_mount_test_dev + here=$(pwd) + cd "$TEST_MNT" || _fail "cannot cd to TEST_MNT" + # ext2_saved/image must not be deleted + run_mayfail $SUDO_HELPER find "$TEST_MNT"/ -mindepth 1 -path '*ext2_saved' -prune -o -exec rm -vrf "{}" \; + cd "$here" + run_check $TOP/btrfs filesystem sync "$TEST_MNT" + run_check_umount_test_dev + convert_test_post_rollback + convert_test_post_check "$CHECKSUMTMP" + + # mount again and verify checksums + convert_test_post_check "$CHECKSUMTMP" + rm "$CHECKSUMTMP" +} + +for feature in '' 'extref' 'skinny-metadata' 'no-holes'; do + do_test "$feature" "ext4 4k nodesize" 4096 mke2fs -t ext4 -b 4096 + do_test "$feature" "ext4 8k nodesize" 8192 mke2fs -t ext4 -b 4096 + do_test "$feature" "ext4 16k nodesize" 16384 mke2fs -t ext4 -b 4096 + do_test "$feature" "ext4 32k nodesize" 32768 mke2fs -t ext4 -b 4096 + do_test "$feature" "ext4 64k nodesize" 65536 mke2fs -t ext4 -b 4096 +done diff --git a/tests/fsck-tests.sh b/tests/fsck-tests.sh new file mode 100755 index 00000000..d1cd7329 --- /dev/null +++ b/tests/fsck-tests.sh @@ -0,0 +1,64 @@ +#!/bin/bash +# +# loop through all of our bad images and make sure fsck repairs them properly + +LANG=C +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +TEST_DEV=${TEST_DEV:-} +RESULTS="$TOP/tests/fsck-tests-results.txt" +IMAGE="$TOP/tests/test.img" + +source $TOP/tests/common + +export TOP +export RESULTS +export LANG +export IMAGE +export TEST_DEV + +rm -f $RESULTS + +# test rely on corrupting blocks tool +check_prereq btrfs-corrupt-block +check_prereq btrfs-image +check_prereq btrfs + +run_one_test() { + local testname + + testname="$1" + echo " [TEST/fsck] $(basename $testname)" + cd $testname + echo "=== Entering $testname" >> $RESULTS + if [ -x test.sh ]; then + # Type 2 + ./test.sh + if [ $? -ne 0 ]; then + _fail "test failed for case $(basename $testname)" + fi + else + # Type 1 + check_all_images `pwd` + fi + cd $TOP +} + +# Each dir contains one type of error for btrfsck test. +# Each dir must be one of the following 2 types: +# 1) Only btrfs-image dump +# Only contains one or several btrfs-image dumps (.img) +# Each image will be tested by generic test routine +# (btrfsck --repair and btrfsck). +# This is for case that btree-healthy images. +# 2) Custom test script +# This dir contains test.sh which will do custom image +# generation/check/verification. +# This is for case btrfs-image can't dump or case needs extra +# check/verify + +for i in $(find $TOP/tests/fsck-tests -maxdepth 1 -mindepth 1 -type d \ + ${TEST:+-name "$TEST"} | sort) +do + run_one_test "$i" +done diff --git a/tests/fsck-tests/001-bad-file-extent-bytenr/default_case.img b/tests/fsck-tests/001-bad-file-extent-bytenr/default_case.img Binary files differnew file mode 100644 index 00000000..d2a05bb8 --- /dev/null +++ b/tests/fsck-tests/001-bad-file-extent-bytenr/default_case.img diff --git a/tests/fsck-tests/002-bad-transid/default_case.img b/tests/fsck-tests/002-bad-transid/default_case.img Binary files differnew file mode 100644 index 00000000..85bd87cd --- /dev/null +++ b/tests/fsck-tests/002-bad-transid/default_case.img diff --git a/tests/fsck-tests/003-shift-offsets/default_case.img b/tests/fsck-tests/003-shift-offsets/default_case.img Binary files differnew file mode 100644 index 00000000..ce23f673 --- /dev/null +++ b/tests/fsck-tests/003-shift-offsets/default_case.img diff --git a/tests/fsck-tests/004-no-dir-index/default_case.img b/tests/fsck-tests/004-no-dir-index/default_case.img Binary files differnew file mode 100644 index 00000000..6f2483e6 --- /dev/null +++ b/tests/fsck-tests/004-no-dir-index/default_case.img diff --git a/tests/fsck-tests/005-bad-item-offset/default_case.img b/tests/fsck-tests/005-bad-item-offset/default_case.img Binary files differnew file mode 100644 index 00000000..e11e1e32 --- /dev/null +++ b/tests/fsck-tests/005-bad-item-offset/default_case.img diff --git a/tests/fsck-tests/006-bad-root-items/default_case.tar.xz b/tests/fsck-tests/006-bad-root-items/default_case.tar.xz Binary files differnew file mode 100644 index 00000000..125d8e7d --- /dev/null +++ b/tests/fsck-tests/006-bad-root-items/default_case.tar.xz diff --git a/tests/fsck-tests/006-bad-root-items/skinny_case.tar.xz b/tests/fsck-tests/006-bad-root-items/skinny_case.tar.xz Binary files differnew file mode 100644 index 00000000..ed99dc4d --- /dev/null +++ b/tests/fsck-tests/006-bad-root-items/skinny_case.tar.xz diff --git a/tests/fsck-tests/006-bad-root-items/test.sh b/tests/fsck-tests/006-bad-root-items/test.sh new file mode 100755 index 00000000..84332348 --- /dev/null +++ b/tests/fsck-tests/006-bad-root-items/test.sh @@ -0,0 +1,17 @@ +#!/bin/bash + +source $TOP/tests/common + +check_prereq btrfs + +echo "extracting image default_case.tar.xz" >> $RESULTS +tar --no-same-owner -xJf default_case.tar.xz || \ + _fail "failed to extract default_case.tar.xz" +check_image test.img + +echo "extracting image skinny_case.tar.xz" >> $RESULTS +tar --no-same-owner -xJf skinny_case.tar.xz || \ + _fail "failed to extract skinny_case.tar.xz" +check_image test.img + +rm test.img diff --git a/tests/fsck-tests/007-bad-offset-snapshots/default_case.img b/tests/fsck-tests/007-bad-offset-snapshots/default_case.img Binary files differnew file mode 100644 index 00000000..b87e9977 --- /dev/null +++ b/tests/fsck-tests/007-bad-offset-snapshots/default_case.img diff --git a/tests/fsck-tests/008-bad-dir-index-name/default_case.img b/tests/fsck-tests/008-bad-dir-index-name/default_case.img Binary files differnew file mode 100644 index 00000000..e004737f --- /dev/null +++ b/tests/fsck-tests/008-bad-dir-index-name/default_case.img diff --git a/tests/fsck-tests/009-no-dir-item-or-index/default_case.img b/tests/fsck-tests/009-no-dir-item-or-index/default_case.img Binary files differnew file mode 100644 index 00000000..d7f22692 --- /dev/null +++ b/tests/fsck-tests/009-no-dir-item-or-index/default_case.img diff --git a/tests/fsck-tests/010-no-rootdir-inode-item/default_case.img b/tests/fsck-tests/010-no-rootdir-inode-item/default_case.img Binary files differnew file mode 100644 index 00000000..6b66fdad --- /dev/null +++ b/tests/fsck-tests/010-no-rootdir-inode-item/default_case.img diff --git a/tests/fsck-tests/011-no-inode-item/default_case.img b/tests/fsck-tests/011-no-inode-item/default_case.img Binary files differnew file mode 100644 index 00000000..352fc021 --- /dev/null +++ b/tests/fsck-tests/011-no-inode-item/default_case.img diff --git a/tests/fsck-tests/012-leaf-corruption/no_data_extent.tar.xz b/tests/fsck-tests/012-leaf-corruption/no_data_extent.tar.xz Binary files differnew file mode 100644 index 00000000..547e5455 --- /dev/null +++ b/tests/fsck-tests/012-leaf-corruption/no_data_extent.tar.xz diff --git a/tests/fsck-tests/012-leaf-corruption/test.sh b/tests/fsck-tests/012-leaf-corruption/test.sh new file mode 100755 index 00000000..a308727d --- /dev/null +++ b/tests/fsck-tests/012-leaf-corruption/test.sh @@ -0,0 +1,117 @@ +#!/bin/bash + +source $TOP/tests/common + +check_prereq btrfs-image + +# Check file list for leaf corruption, no regular/preallocated +# file extent case. +# Corrupted leaf is 20832256, which contains inode 1862~1872 +# +# 1862, ref from leaf 20828160 key 24(DIR_ITEM) +# 1863, ref from leaf 605388 item key 11(DIR_ITEM) +# 1864, no ref to rebuild, no need to rebuild +# 1865, ref from leaf 19767296 key 23(DIR_ITEM) +# 1866-1868 no ref to rebuild, all refs in corrupted leaf +# 1869, ref from leaf 4976640 key 22(DIR_ITEM) +# 1870 no ref to rebuild, all refs in corrupted leaf +# 1871, ref from leaf 19746816 key 38(DIR_ITEM) +# 1872, ref from leaf 19767296 key 14(DIR_ITEM) +# The list format is: +# INO SIZE MODE NAME +# INO: inode number +# SIZE: file size, only checked for regular file +# MODE: raw file mode, get from stat +# NAME: file name +leaf_no_data_ext_list=( + 1862 0 40700 "install.d" + 1862 0 40700 "install.d" + 1863 0 40700 "gdb" + 1865 0 40700 "iptables" + 1869 0 40700 "snmp" + 1871 0 100700 "machine-id" + 1872 0 100700 "adjtime" + 1877 0 40700 "del" +) + +generate_leaf_corrupt_no_data_ext() +{ + dest=$1 + echo "generating leaf_corrupt_no_data_ext.btrfs-image" >> $RESULTS + tar --no-same-owner -xJf ./no_data_extent.tar.xz || \ + _fail "failed to extract leaf_corrupt_no_data_ext.btrfs-image" + $TOP/btrfs-image -r test.img.btrfs-image $dest || \ + _fail "failed to extract leaf_corrupt_no_data_ext.btrfs-image" + + # leaf at 4206592 and 20905984 contains no regular data + # extent, clear its csum to corrupt the leaf. + for x in 4206592 20905984; do + dd if=/dev/zero of=$dest bs=1 count=32 conv=notrunc seek=$x \ + 1>/dev/null 2>&1 + done +} + +check_inode() +{ + path=$1 + ino=$2 + size=$3 + mode=$4 + name=$5 + + # Check whether the inode exists + exists=$($SUDO_HELPER find $path -inum $ino) + if [ -z "$exists" ]; then + _fail "inode $ino not recovered correctly" + fi + + # Check inode type + found_mode=$(printf "%o" 0x$($SUDO_HELPER stat $exists -c %f)) + if [ $found_mode -ne $mode ]; then + echo "$found_mode" + _fail "inode $ino modes not recovered" + fi + + # Check inode size + found_size=$($SUDO_HELPER stat $exists -c %s) + if [ $mode -ne 41700 -a $found_size -ne $size ]; then + _fail "inode $ino size not recovered correctly" + fi + + # Check inode name + if [ "$(basename $exists)" != "$name" ]; then + _fail "inode $ino name not recovered correctly" + else + return 0 + fi +} + +# Check salvaged data in the recovered image +check_leaf_corrupt_no_data_ext() +{ + image=$1 + $SUDO_HELPER mount -o loop $image -o ro $TEST_MNT + + i=0 + while [ $i -lt ${#leaf_no_data_ext_list[@]} ]; do + check_inode $TEST_MNT/lost+found \ + ${leaf_no_data_ext_list[i]} \ + ${leaf_no_data_ext_list[i + 1]} \ + ${leaf_no_data_ext_list[i + 2]} \ + ${leaf_no_data_ext_list[i + 3]} \ + ${leaf_no_data_ext_list[i + 4]} + ((i+=4)) + done + $SUDO_HELPER umount $TEST_MNT +} + +setup_root_helper + +generate_leaf_corrupt_no_data_ext test.img +check_image test.img +check_leaf_corrupt_no_data_ext test.img + +rm test.img +rm test.img.btrfs-image +# Not used, its function is the same as generate_leaf_corrupt_no_data_ext() +rm generate_image.sh diff --git a/tests/fsck-tests/013-extent-tree-rebuild/test.sh b/tests/fsck-tests/013-extent-tree-rebuild/test.sh new file mode 100755 index 00000000..ff7d28e5 --- /dev/null +++ b/tests/fsck-tests/013-extent-tree-rebuild/test.sh @@ -0,0 +1,44 @@ +#!/bin/bash + +source $TOP/tests/common + +check_prereq btrfs-corrupt-block +check_prereq btrfs-debug-tree +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper +prepare_test_dev 1G + +# test whether fsck can rebuild a corrupted extent tree +test_extent_tree_rebuild() +{ + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f $TEST_DEV + + run_check_mount_test_dev + run_check $SUDO_HELPER cp -aR /lib/modules/`uname -r`/ $TEST_MNT + + for i in `seq 1 100`;do + run_check $SUDO_HELPER $TOP/btrfs sub snapshot $TEST_MNT \ + $TEST_MNT/snapaaaaaaa_$i + done + run_check_umount_test_dev + + # get extent root bytenr + extent_root_bytenr=`$SUDO_HELPER $TOP/btrfs-debug-tree -r $TEST_DEV | \ + grep extent | awk '{print $7}'` + if [ -z $extent_root_bytenr ];then + _fail "fail to get extent root bytenr" + fi + + # corrupt extent root node block + run_check $SUDO_HELPER $TOP/btrfs-corrupt-block -l $extent_root_bytenr \ + -b 4096 $TEST_DEV + + $SUDO_HELPER $TOP/btrfs check $TEST_DEV >& /dev/null && \ + _fail "btrfs check should detect failure" + run_check $SUDO_HELPER $TOP/btrfs check --init-extent-tree $TEST_DEV + run_check $SUDO_HELPER $TOP/btrfs check $TEST_DEV +} + +test_extent_tree_rebuild diff --git a/tests/fsck-tests/014-no-extent-info/default_case.img b/tests/fsck-tests/014-no-extent-info/default_case.img Binary files differnew file mode 100644 index 00000000..1ff27434 --- /dev/null +++ b/tests/fsck-tests/014-no-extent-info/default_case.img diff --git a/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.txt b/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.txt new file mode 100644 index 00000000..9685ed46 --- /dev/null +++ b/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.txt @@ -0,0 +1,254 @@ +URL: https://bugzilla.kernel.org/show_bug.cgi?id=97171 + +The btrfs-image attached to this bug causes the btrfs-userland tool to use +uninitialized memory and ultimately overwrite what seems to be arbitrary memory +locations, dying in the process. Reproduced on x86-64 and i686. + +The kernel seems to be less affected and fails to mount the image. If +/usr/sbin/btrfs is not setuid (which it probably never is), things should be +safe. I didn't investigate further though. + +gdb output: + +GNU gdb (GDB) Fedora 7.8.2-38.fc21 +[... lots of other errors...] +Ignoring transid failure +root 5 inode 260 errors 1000, some csum missing + unresolved ref dir 256 index 7 namelen 5 name b.bin filetype 1 errors 2, no dir index + unresolved ref dir 256 index 7 namelen 5 name b.fin filetype 1 errors 5, no dir item, no inode ref +root 5 inode 261 errors 200, dir isize wrong + +Program received signal SIGSEGV, Segmentation fault. +0x000000000089bb70 in ?? () +(gdb) bt +#0 0x000000000089bb70 in ?? () +#1 0x00007fffffffdb50 in ?? () +#2 0x0000000000894b20 in ?? () +#3 0x00000032629b88e0 in _IO_2_1_stdout_ () from /lib64/libc.so.6 +#4 0x000000000088c010 in ?? () +#5 0x0000000000000000 in ?? () + + +valgrind output: + +[...lots of errors...] +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x436E77: check_block.part.14 (ctree.c:548) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x4A0B0E7: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x4A0B2AC: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x4A0B151: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x4A0B162: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x4A0B176: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x4A0B2CE: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Conditional jump or move depends on uninitialised value(s) +==12638== at 0x4A0B34A: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Use of uninitialised value of size 8 +==12638== at 0x4A0B3A0: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== +==12638== Invalid read of size 1 +==12638== at 0x4A0B3A0: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== Address 0xa25c9de9 is not stack'd, malloc'd or (recently) free'd +==12638== +==12638== +==12638== Process terminating with default action of signal 11 (SIGSEGV) +==12638== Access not within mapped region at address 0xA25C9DE9 +==12638== at 0x4A0B3A0: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==12638== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==12638== by 0x436E99: check_block.part.14 (ctree.c:550) +==12638== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==12638== by 0x438954: btrfs_search_slot (ctree.c:1120) +==12638== by 0x40DD1F: count_csum_range (cmds-check.c:1419) +==12638== by 0x40DD1F: process_file_extent (cmds-check.c:1551) +==12638== by 0x40DD1F: process_one_leaf (cmds-check.c:1617) +==12638== by 0x40DD1F: walk_down_tree (cmds-check.c:1742) +==12638== by 0x40DD1F: check_fs_root (cmds-check.c:3380) +==12638== by 0x40DD1F: check_fs_roots.isra.51 (cmds-check.c:3516) +==12638== by 0x4C64B0F: ??? +==12638== by 0x4C30A2F: ??? +==12638== by 0x4C468CF: ??? +==12638== by 0x32629B88DF: ??? (in /usr/lib64/libc-2.20.so) +==12638== by 0x4C3657F: ??? +==12638== If you believe this happened as a result of a stack +==12638== overflow in your program's main thread (unlikely but +==12638== possible), you can try to increase the size of the +==12638== main thread stack using the --main-stacksize= flag. +==12638== The main thread stack size used in this run was 8388608. +==12638== +==12638== HEAP SUMMARY: +==12638== in use at exit: 46,260 bytes in 56 blocks +==12638== total heap usage: 380 allocs, 324 frees, 218,054 bytes allocated +==12638== +==12638== LEAK SUMMARY: +==12638== definitely lost: 272 bytes in 2 blocks +==12638== indirectly lost: 800 bytes in 8 blocks +==12638== possibly lost: 88 bytes in 1 blocks +==12638== still reachable: 45,100 bytes in 45 blocks +==12638== suppressed: 0 bytes in 0 blocks +==12638== Rerun with --leak-check=full to see details of leaked memory +==12638== +==12638== For counts of detected and suppressed errors, rerun with: -v +==12638== Use --track-origins=yes to see where uninitialised values come from +==12638== ERROR SUMMARY: 10 errors from 10 contexts (suppressed: 0 from 0) +[1] 12638 segmentation fault valgrind btrfs check btrfs_fukked_memorycorruption.bin diff --git a/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.xz b/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.xz Binary files differnew file mode 100644 index 00000000..f3f0944d --- /dev/null +++ b/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.xz diff --git a/tests/fsck-tests/016-wrong-inode-nbytes/default_case.img.xz b/tests/fsck-tests/016-wrong-inode-nbytes/default_case.img.xz Binary files differnew file mode 100644 index 00000000..d513acf5 --- /dev/null +++ b/tests/fsck-tests/016-wrong-inode-nbytes/default_case.img.xz diff --git a/tests/fsck-tests/017-missing-all-file-extent/default_case.img.xz b/tests/fsck-tests/017-missing-all-file-extent/default_case.img.xz Binary files differnew file mode 100644 index 00000000..10cd4c78 --- /dev/null +++ b/tests/fsck-tests/017-missing-all-file-extent/default_case.img.xz diff --git a/tests/fsck-tests/018-leaf-crossing-stripes/default_case.raw.xz b/tests/fsck-tests/018-leaf-crossing-stripes/default_case.raw.xz Binary files differnew file mode 100644 index 00000000..60eb2f97 --- /dev/null +++ b/tests/fsck-tests/018-leaf-crossing-stripes/default_case.raw.xz diff --git a/tests/fsck-tests/018-leaf-crossing-stripes/test.sh b/tests/fsck-tests/018-leaf-crossing-stripes/test.sh new file mode 100755 index 00000000..c453ab5c --- /dev/null +++ b/tests/fsck-tests/018-leaf-crossing-stripes/test.sh @@ -0,0 +1,12 @@ +#!/bin/bash + +source $TOP/tests/common + +check_prereq btrfs + +image=$(extract_image "./default_case.raw.xz") +run_check_stdout $TOP/btrfs check "$image" 2>&1 | + grep -q "crossing stripe boundary" || + _fail "no expected error message in the output" + +rm -f "$image" diff --git a/tests/fsck-tests/019-non-skinny-false-alert/default_case.img.xz b/tests/fsck-tests/019-non-skinny-false-alert/default_case.img.xz Binary files differnew file mode 100644 index 00000000..c35f8bc6 --- /dev/null +++ b/tests/fsck-tests/019-non-skinny-false-alert/default_case.img.xz diff --git a/tests/fsck-tests/019-non-skinny-false-alert/test.sh b/tests/fsck-tests/019-non-skinny-false-alert/test.sh new file mode 100755 index 00000000..a7f8e862 --- /dev/null +++ b/tests/fsck-tests/019-non-skinny-false-alert/test.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# $ btrfs check img +# Checking filesystem on img +# UUID: 17f2bf15-f4c2-4ebc-b1f7-39b7af26257a +# checking extents +# bad extent [29376512, 29392896), type mismatch with chunk +# bad extent [29442048, 29458432), type mismatch with chunk +# bad extent [29589504, 29605888), type mismatch with chunk +# ... +# +# a buggy check leads to the above messages + +source $TOP/tests/common + +check_prereq btrfs + +image=$(extract_image "./default_case.img.xz") +run_check_stdout $TOP/btrfs check "$image" 2>&1 | + grep -q "type mismatch with chunk" && + _fail "unexpected error message in the output" + +rm -f "$image" diff --git a/tests/fsck-tests/020-extent-ref-cases/keyed_block_ref.img b/tests/fsck-tests/020-extent-ref-cases/keyed_block_ref.img Binary files differnew file mode 100644 index 00000000..289d37bc --- /dev/null +++ b/tests/fsck-tests/020-extent-ref-cases/keyed_block_ref.img diff --git a/tests/fsck-tests/020-extent-ref-cases/keyed_data_ref.img b/tests/fsck-tests/020-extent-ref-cases/keyed_data_ref.img Binary files differnew file mode 100644 index 00000000..2ac0ae53 --- /dev/null +++ b/tests/fsck-tests/020-extent-ref-cases/keyed_data_ref.img diff --git a/tests/fsck-tests/020-extent-ref-cases/no_inline_ref.img b/tests/fsck-tests/020-extent-ref-cases/no_inline_ref.img Binary files differnew file mode 100644 index 00000000..b05ae73f --- /dev/null +++ b/tests/fsck-tests/020-extent-ref-cases/no_inline_ref.img diff --git a/tests/fsck-tests/020-extent-ref-cases/no_skinny_ref.img b/tests/fsck-tests/020-extent-ref-cases/no_skinny_ref.img Binary files differnew file mode 100644 index 00000000..900b65ca --- /dev/null +++ b/tests/fsck-tests/020-extent-ref-cases/no_skinny_ref.img diff --git a/tests/fsck-tests/020-extent-ref-cases/shared_block_ref.img b/tests/fsck-tests/020-extent-ref-cases/shared_block_ref.img Binary files differnew file mode 100644 index 00000000..8d7b50f7 --- /dev/null +++ b/tests/fsck-tests/020-extent-ref-cases/shared_block_ref.img diff --git a/tests/fsck-tests/020-extent-ref-cases/shared_data_ref.img b/tests/fsck-tests/020-extent-ref-cases/shared_data_ref.img Binary files differnew file mode 100644 index 00000000..aa2dafa5 --- /dev/null +++ b/tests/fsck-tests/020-extent-ref-cases/shared_data_ref.img diff --git a/tests/fsck-tests/020-extent-ref-cases/test.sh b/tests/fsck-tests/020-extent-ref-cases/test.sh new file mode 100755 index 00000000..c2b6a006 --- /dev/null +++ b/tests/fsck-tests/020-extent-ref-cases/test.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# In order to confirm that btrfsck supports to check a variety of refs, add the +# following cases: +# +# * keyed_block_ref +# * keyed_data_ref +# * shared_block_ref +# * shared_data_ref +# * no_inline_ref (a extent item without inline ref) +# * no_skinny_ref + +source $TOP/tests/common + +check_prereq btrfs + +for img in *.img +do + image=$(extract_image $img) + run_check_stdout $TOP/btrfs check "$image" 2>&1 | + grep -q "Errors found in extent allocation tree or chunk allocation" && + _fail "unexpected error occurred when checking $img" + rm -f "$image" +done diff --git a/tests/fuzz-tests.sh b/tests/fuzz-tests.sh new file mode 100755 index 00000000..29691cae --- /dev/null +++ b/tests/fuzz-tests.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# +# misc tests on fuzzed or crafted images + +LANG=C +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +TEST_DEV=${TEST_DEV:-} +RESULTS="$TOP/tests/fuzz-tests-results.txt" +IMAGE="$TOP/tests/test.img" + +source $TOP/tests/common + +export TOP +export RESULTS +export LANG +export IMAGE +export TEST_DEV + +rm -f $RESULTS + +check_prereq btrfs + +# The tests are driven by their custom script called 'test.sh' + +for i in $(find $TOP/tests/fuzz-tests -maxdepth 1 -mindepth 1 -type d \ + ${TEST:+-name "$TEST"} | sort) +do + name=$(basename $i) + cd $i + if [ -x test.sh ]; then + echo "=== Entering $i" >> $RESULTS + echo " [TEST/fuzz] $name" + ./test.sh + if [ $? -ne 0 ]; then + _fail "test failed for case $(basename $i)" + fi + fi + cd $TOP +done diff --git a/tests/fuzz-tests/001-simple-unmounted/test.sh b/tests/fuzz-tests/001-simple-unmounted/test.sh new file mode 100755 index 00000000..bf01a3a4 --- /dev/null +++ b/tests/fuzz-tests/001-simple-unmounted/test.sh @@ -0,0 +1,20 @@ +#!/bin/sh + +# iterate over all fuzzed images and run 'btrfs check' + +source $TOP/tests/common + +setup_root_helper +check_prereq btrfs + +# redefine the one provided by common +check_image() { + local image + + image=$1 + run_mayfail $TOP/btrfs check "$image" +} + +check_all_images $TOP/tests/fuzz-tests/images + +exit 0 diff --git a/tests/fuzz-tests/images/bad-superblock-1.raw.xz b/tests/fuzz-tests/images/bad-superblock-1.raw.xz Binary files differnew file mode 100644 index 00000000..3d6358f0 --- /dev/null +++ b/tests/fuzz-tests/images/bad-superblock-1.raw.xz diff --git a/tests/fuzz-tests/images/bad-superblock-2.raw.xz b/tests/fuzz-tests/images/bad-superblock-2.raw.xz Binary files differnew file mode 100644 index 00000000..7db7610b --- /dev/null +++ b/tests/fuzz-tests/images/bad-superblock-2.raw.xz diff --git a/tests/fuzz-tests/images/bad-superblock-3.raw.xz b/tests/fuzz-tests/images/bad-superblock-3.raw.xz Binary files differnew file mode 100644 index 00000000..4aa31483 --- /dev/null +++ b/tests/fuzz-tests/images/bad-superblock-3.raw.xz diff --git a/tests/fuzz-tests/images/bad-superblock.txt b/tests/fuzz-tests/images/bad-superblock.txt new file mode 100644 index 00000000..f7dd9aa0 --- /dev/null +++ b/tests/fuzz-tests/images/bad-superblock.txt @@ -0,0 +1,17 @@ +bad-superblock-*.txt + +Crafted images from Jiri Slaby, produced by some symbolic execution framework +that finds unhandled cases at mount time. + +Relevant kernel patches to backport: + +e3540eab29e1b2260bc4b9b3979a49a00e3e3af8 +btrfs: add more checks to btrfs_read_sys_array + +ce7fca5f57ed0fcd7e7b3d7b1a3e1791f8e56fa3 +btrfs: add checks for sys_chunk_array sizes + +75d6ad382bb91f363452119d34238e156589ca2d +btrfs: more superblock checks, lower bounds on devices and sectorsize/nodesize + +(and more from fs/btrfs/super.c) diff --git a/tests/fuzz-tests/images/bko-104131-fsck-oob-read.raw.xz b/tests/fuzz-tests/images/bko-104131-fsck-oob-read.raw.xz Binary files differnew file mode 100644 index 00000000..7848f8b1 --- /dev/null +++ b/tests/fuzz-tests/images/bko-104131-fsck-oob-read.raw.xz diff --git a/tests/fuzz-tests/images/bko-104131-fsck-oob-read.txt b/tests/fuzz-tests/images/bko-104131-fsck-oob-read.txt new file mode 100644 index 00000000..0e829c2e --- /dev/null +++ b/tests/fuzz-tests/images/bko-104131-fsck-oob-read.txt @@ -0,0 +1,31 @@ +URL: https://bugzilla.kernel.org/show_bug.cgi?id=104131 +Hanno Boeck 2015-09-07 07:24:32 UTC + +Created attachment 186941 [details] +malformed btrfs filesystem causing oob read + +The attached malformed filesystem image will cause an invalid heap out of bounds memory read in btrfsck. + +This was found while fuzzing btrfs-progs with american fuzzy lop. + +Stack trace from Address Sanitizer: +==31289==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60f00000f003 at pc 0x0000005d0dbb bp 0x7ffdf444c180 sp 0x7ffdf444c178 +READ of size 8 at 0x60f00000f003 thread T0 + #0 0x5d0dba in btrfs_header_bytenr /mnt/ram/btrfs-progs-v4.1.2/./ctree.h:1797:1 + #1 0x5d0dba in check_tree_block /mnt/ram/btrfs-progs-v4.1.2/disk-io.c:60 + #2 0x5d0dba in read_tree_block /mnt/ram/btrfs-progs-v4.1.2/disk-io.c:337 + #3 0x5dc00e in btrfs_setup_chunk_tree_and_device_map /mnt/ram/btrfs-progs-v4.1.2/disk-io.c:1169:30 + #4 0x5dcf89 in __open_ctree_fd /mnt/ram/btrfs-progs-v4.1.2/disk-io.c:1261:8 + #5 0x5dc50a in open_ctree_fs_info /mnt/ram/btrfs-progs-v4.1.2/disk-io.c:1302:9 + #6 0x52f22f in cmd_check /mnt/ram/btrfs-progs-v4.1.2/cmds-check.c:9333:9 + #7 0x4e7bcc in main /mnt/ram/btrfs-progs-v4.1.2/btrfs.c:245:7 + #8 0x7f98bb101f9f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.20-r2/work/glibc-2.20/csu/libc-start.c:289 + #9 0x41f748 in _start (/mnt/ram/btrfs/btrfs+0x41f748) + +0x60f00000f003 is located 3 bytes to the right of 176-byte region [0x60f00000ef50,0x60f00000f000) +allocated by thread T0 here: + #0 0x4bade8 in malloc (/mnt/ram/btrfs/btrfs+0x4bade8) + #1 0x622c24 in __alloc_extent_buffer /mnt/ram/btrfs-progs-v4.1.2/extent_io.c:541:7 + #2 0x622c24 in alloc_extent_buffer /mnt/ram/btrfs-progs-v4.1.2/extent_io.c:648 + #3 0x5cf436 in btrfs_find_create_tree_block /mnt/ram/btrfs-progs-v4.1.2/disk-io.c:186:9 + #4 0x5cf436 in read_tree_block /mnt/ram/btrfs-progs-v4.1.2/disk-io.c:314 diff --git a/tests/fuzz-tests/images/bko-104141-fsck-exception.raw.xz b/tests/fuzz-tests/images/bko-104141-fsck-exception.raw.xz Binary files differnew file mode 100644 index 00000000..d24a32f8 --- /dev/null +++ b/tests/fuzz-tests/images/bko-104141-fsck-exception.raw.xz diff --git a/tests/fuzz-tests/images/bko-104141-fsck-exception.txt b/tests/fuzz-tests/images/bko-104141-fsck-exception.txt new file mode 100644 index 00000000..aed91909 --- /dev/null +++ b/tests/fuzz-tests/images/bko-104141-fsck-exception.txt @@ -0,0 +1,9 @@ +URL: https://bugzilla.kernel.org/show_bug.cgi?id=104141 +Hanno Boeck 2015-09-07 07:27:58 UTC + +Created attachment 186951 [details] +malformed filesystem causing floating point exception + +The attacked file will cause a floating point exception in btrfsck. + +This was found while fuzzing btrfs-progs with american fuzzy lop. diff --git a/tests/fuzz-tests/images/bko-96971-btrfs-image.raw.xz b/tests/fuzz-tests/images/bko-96971-btrfs-image.raw.xz Binary files differnew file mode 100644 index 00000000..21aa33b0 --- /dev/null +++ b/tests/fuzz-tests/images/bko-96971-btrfs-image.raw.xz diff --git a/tests/fuzz-tests/images/bko-96971-btrfs-image.txt b/tests/fuzz-tests/images/bko-96971-btrfs-image.txt new file mode 100644 index 00000000..ff85540d --- /dev/null +++ b/tests/fuzz-tests/images/bko-96971-btrfs-image.txt @@ -0,0 +1,69 @@ +URL: https://bugzilla.kernel.org/show_bug.cgi?id=96971 + Lukas Lueg 2015-04-20 23:01:44 UTC + +I've identified some problems in the btrfs code and attached a btrfs-image +which causes the userland tools to crash and the kernel to immediately freeze +once the filesystem get's mounted and one of the files is accessed. Putting +the image onto a usb-drive gives you a freeze-on-a-stick :-) + +"btrfs check" crashes due to a SIGFPE in count_csum_range(). The culprit is +struct btrfs_root->fs_info->super_copy->csum_size being 0, which goes +unchecked before entering a division. I was not able to identify where the +kernel crashes (system goes down the tubes), yet the problem is probably the +same. + +"btrfs version" is v3.19.1; bug is also present in latest git (kdave and +unstable) as of 2015/04/21 + + +Full gdb output: + +gdb btrfs +GNU gdb (GDB) Fedora 7.8.2-38.fc21 +Copyright (C) 2014 Free Software Foundation, Inc. +License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. Type "show copying" +and "show warranty" for details. +This GDB was configured as "x86_64-redhat-linux-gnu". +Type "show configuration" for configuration details. +For bug reporting instructions, please see: +<http://www.gnu.org/software/gdb/bugs/>. +Find the GDB manual and other documentation resources online at: +<http://www.gnu.org/software/gdb/documentation/>. +For help, type "help". +Type "apropos word" to search for commands related to "word"... +Reading symbols from btrfs...Reading symbols from /usr/lib/debug/usr/sbin/btrfs.debug...done. +done. +(gdb) run check btrfs_fukked.bin +Starting program: /usr/sbin/btrfs check btrfs_fukked.bin +[Thread debugging using libthread_db enabled] +Using host libthread_db library "/lib64/libthread_db.so.1". +Checking filesystem on btrfs_fukked.bin +UUID: cdd8684f-9eb1-40a4-91ec-1ed7c3cb444c +checking extents +checking free space cache +checking fs roots + +Program received signal SIGFPE, Arithmetic exception. +count_csum_range (root=<optimized out>, root=<optimized out>, + found=<synthetic pointer>, len=7385088, start=7471104) at cmds-check.c:1455 +1455 csum_end = key.offset + (size / csum_size) * root->sectorsize; +(gdb) bt +#0 count_csum_range (root=<optimized out>, root=<optimized out>, + found=<synthetic pointer>, len=7385088, start=7471104) at cmds-check.c:1455 +#1 process_file_extent (active_node=0x7fffffffd710, key=0x7fffffffd680, + slot=11, eb=<optimized out>, root=0x894b10) at cmds-check.c:1551 +#2 process_one_leaf (wc=0x7fffffffd6c0, eb=<optimized out>, root=0x894b10) + at cmds-check.c:1617 +#3 walk_down_tree (level=<synthetic pointer>, wc=0x7fffffffd6c0, + path=0x7fffffffd7f0, root=0x894b10) at cmds-check.c:1742 +#4 check_fs_root (wc=0x7fffffffd6c0, root_cache=0x7fffffffdb20, root=0x894b10) + at cmds-check.c:3380 +#5 check_fs_roots (root_cache=root_cache@entry=0x7fffffffdb20, root=0x894b10) + at cmds-check.c:3516 +#6 0x0000000000428aea in cmd_check (argc=<optimized out>, + argv=<optimized out>) at cmds-check.c:9465 +#7 0x000000000040e5a2 in main (argc=2, argv=0x7fffffffdeb0) at btrfs.c:245 +(gdb) p csum_size +$2 = 0 diff --git a/tests/fuzz-tests/images/bko-97191-btrfs-image.raw.txt b/tests/fuzz-tests/images/bko-97191-btrfs-image.raw.txt new file mode 100644 index 00000000..f0d81894 --- /dev/null +++ b/tests/fuzz-tests/images/bko-97191-btrfs-image.raw.txt @@ -0,0 +1,137 @@ +URL: https://bugzilla.kernel.org/show_bug.cgi?id=97191 +Lukas Lueg 2015-04-23 22:20:35 UTC + +Running btrfs-progs v3.19.1 + +The btrfs-image attached to this bug causes the btrfs-userland tool to +overflow some data structures, leading to unallocated memory being written to +and read from. A segfault results shortly after. Reproduced on x86-64 and +i686. + +The kernel seems to be less affected and fails to mount the image. I didn't +investigate whether the reads/writes could be used to gain control over $EIP. +Since the first invalid write of 8 bytes seems to run into adjacent heap +blocks (crash in unlink()), it may be possible though. + +gdb output: + +Program received signal SIGSEGV, Segmentation fault. +malloc_consolidate (av=av@entry=0x32629b7cc0 <main_arena>) at malloc.c:4151 +4151 unlink(av, p, bck, fwd); +(gdb) bt +#0 malloc_consolidate (av=av@entry=0x32629b7cc0 <main_arena>) at malloc.c:4151 +#1 0x0000003262680628 in _int_malloc (av=av@entry=0x32629b7cc0 <main_arena>, bytes=bytes@entry=4224) at malloc.c:3420 +#2 0x000000326268315e in __GI___libc_malloc (bytes=4224) at malloc.c:2896 +#3 0x0000000000449d15 in __alloc_extent_buffer (tree=0x88c078, bytenr=4288512, blocksize=4096) at extent_io.c:541 +#4 0x000000000044a8b4 in alloc_extent_buffer (tree=0x88c078, bytenr=4288512, blocksize=4096) at extent_io.c:648 +#5 0x000000000043b1a0 in btrfs_find_create_tree_block (root=root@entry=0x895840, bytenr=<optimized out>, + blocksize=<optimized out>) at disk-io.c:159 +#6 0x000000000043ca4e in read_tree_block (root=root@entry=0x895840, bytenr=<optimized out>, blocksize=<optimized out>, + parent_transid=13) at disk-io.c:287 +#7 0x000000000043ccb7 in find_and_setup_root (tree_root=0x88c250, fs_info=<optimized out>, objectid=5, root=0x895840) + at disk-io.c:557 +#8 0x000000000043ce92 in btrfs_read_fs_root_no_cache (fs_info=fs_info@entry=0x88c010, location=location@entry=0x7fffffffd960) + at disk-io.c:640 +#9 0x000000000043d060 in btrfs_read_fs_root (fs_info=fs_info@entry=0x88c010, location=location@entry=0x7fffffffd960) + at disk-io.c:739 +#10 0x000000000043d48c in btrfs_setup_all_roots (fs_info=fs_info@entry=0x88c010, root_tree_bytenr=<optimized out>, + root_tree_bytenr@entry=0, flags=flags@entry=OPEN_CTREE_EXCLUSIVE) at disk-io.c:988 +#11 0x000000000043d802 in __open_ctree_fd (fp=fp@entry=3, path=path@entry=0x7fffffffe20d "ramdisk/btrfs_fukked.bin", + sb_bytenr=65536, sb_bytenr@entry=0, root_tree_bytenr=root_tree_bytenr@entry=0, flags=flags@entry=OPEN_CTREE_EXCLUSIVE) + at disk-io.c:1199 +#12 0x000000000043d965 in open_ctree_fs_info (filename=0x7fffffffe20d "ramdisk/btrfs_fukked.bin", sb_bytenr=sb_bytenr@entry=0, + root_tree_bytenr=root_tree_bytenr@entry=0, flags=flags@entry=OPEN_CTREE_EXCLUSIVE) at disk-io.c:1231 +#13 0x0000000000427bf5 in cmd_check (argc=1, argv=0x7fffffffdea0) at cmds-check.c:9326 +#14 0x000000000040e5a2 in main (argc=2, argv=0x7fffffffdea0) at btrfs.c:245 + + +valgrind output: + +==32463== Memcheck, a memory error detector +==32463== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. +==32463== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info +==32463== Command: btrfs check ramdisk/btrfs_fukked.bin +==32463== +==32463== Invalid write of size 8 +==32463== at 0x4386FB: btrfs_search_slot (ctree.c:1119) +==32463== by 0x4427F7: UnknownInlinedFun (extent-tree.c:3117) +==32463== by 0x4427F7: btrfs_read_block_groups (extent-tree.c:3167) +==32463== by 0x43D4F2: btrfs_setup_all_roots (disk-io.c:983) +==32463== by 0x43D801: __open_ctree_fd (disk-io.c:1199) +==32463== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==32463== by 0x427BF4: cmd_check (cmds-check.c:9326) +==32463== by 0x40E5A1: main (btrfs.c:245) +==32463== Address 0x4c409f0 is 16 bytes after a block of size 144 alloc'd +==32463== at 0x4A08946: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==32463== by 0x4427AB: btrfs_read_block_groups (extent-tree.c:3162) +==32463== by 0x43D4F2: btrfs_setup_all_roots (disk-io.c:983) +==32463== by 0x43D801: __open_ctree_fd (disk-io.c:1199) +==32463== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==32463== by 0x427BF4: cmd_check (cmds-check.c:9326) +==32463== by 0x40E5A1: main (btrfs.c:245) +==32463== +==32463== Invalid read of size 8 +==32463== at 0x436E70: check_block.part.14 (ctree.c:548) +==32463== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==32463== by 0x438954: btrfs_search_slot (ctree.c:1120) +==32463== by 0x4427F7: UnknownInlinedFun (extent-tree.c:3117) +==32463== by 0x4427F7: btrfs_read_block_groups (extent-tree.c:3167) +==32463== by 0x43D4F2: btrfs_setup_all_roots (disk-io.c:983) +==32463== by 0x43D801: __open_ctree_fd (disk-io.c:1199) +==32463== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==32463== by 0x427BF4: cmd_check (cmds-check.c:9326) +==32463== by 0x40E5A1: main (btrfs.c:245) +==32463== Address 0x4c409f8 is 24 bytes after a block of size 144 in arena "client" +==32463== +==32463== Invalid read of size 4 +==32463== at 0x436E84: UnknownInlinedFun (ctree.h:1605) +==32463== by 0x436E84: UnknownInlinedFun (ctree.h:1612) +==32463== by 0x436E84: check_block.part.14 (ctree.c:550) +==32463== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==32463== by 0x438954: btrfs_search_slot (ctree.c:1120) +==32463== by 0x4427F7: UnknownInlinedFun (extent-tree.c:3117) +==32463== by 0x4427F7: btrfs_read_block_groups (extent-tree.c:3167) +==32463== by 0x43D4F2: btrfs_setup_all_roots (disk-io.c:983) +==32463== by 0x43D801: __open_ctree_fd (disk-io.c:1199) +==32463== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==32463== by 0x427BF4: cmd_check (cmds-check.c:9326) +==32463== by 0x40E5A1: main (btrfs.c:245) +==32463== Address 0x4c409e4 is 4 bytes after a block of size 144 alloc'd +==32463== at 0x4A08946: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==32463== by 0x4427AB: btrfs_read_block_groups (extent-tree.c:3162) +==32463== by 0x43D4F2: btrfs_setup_all_roots (disk-io.c:983) +==32463== by 0x43D801: __open_ctree_fd (disk-io.c:1199) +==32463== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==32463== by 0x427BF4: cmd_check (cmds-check.c:9326) +==32463== by 0x40E5A1: main (btrfs.c:245) +==32463== +==32463== Invalid read of size 1 +==32463== at 0x4A0B3A0: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==32463== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==32463== by 0x436E99: check_block.part.14 (ctree.c:550) +==32463== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==32463== by 0x438954: btrfs_search_slot (ctree.c:1120) +==32463== by 0x4427F7: UnknownInlinedFun (extent-tree.c:3117) +==32463== by 0x4427F7: btrfs_read_block_groups (extent-tree.c:3167) +==32463== by 0x43D4F2: btrfs_setup_all_roots (disk-io.c:983) +==32463== by 0x43D801: __open_ctree_fd (disk-io.c:1199) +==32463== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==32463== by 0x427BF4: cmd_check (cmds-check.c:9326) +==32463== by 0x40E5A1: main (btrfs.c:245) +==32463== Address 0x1b1 is not stack'd, malloc'd or (recently) free'd +==32463== +==32463== +==32463== Process terminating with default action of signal 11 (SIGSEGV) +==32463== Access not within mapped region at address 0x1B1 +==32463== at 0x4A0B3A0: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==32463== by 0x436E99: UnknownInlinedFun (ctree.h:1613) +==32463== by 0x436E99: check_block.part.14 (ctree.c:550) +==32463== by 0x438954: UnknownInlinedFun (kerncompat.h:91) +==32463== by 0x438954: btrfs_search_slot (ctree.c:1120) +==32463== by 0x4427F7: UnknownInlinedFun (extent-tree.c:3117) +==32463== by 0x4427F7: btrfs_read_block_groups (extent-tree.c:3167) +==32463== by 0x43D4F2: btrfs_setup_all_roots (disk-io.c:983) +==32463== by 0x43D801: __open_ctree_fd (disk-io.c:1199) +==32463== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==32463== by 0x427BF4: cmd_check (cmds-check.c:9326) +==32463== by 0x40E5A1: main (btrfs.c:245) diff --git a/tests/fuzz-tests/images/bko-97191-btrfs-image.raw.xz b/tests/fuzz-tests/images/bko-97191-btrfs-image.raw.xz Binary files differnew file mode 100644 index 00000000..b2e48c08 --- /dev/null +++ b/tests/fuzz-tests/images/bko-97191-btrfs-image.raw.xz diff --git a/tests/fuzz-tests/images/bko-97271-btrfs-image.raw.txt b/tests/fuzz-tests/images/bko-97271-btrfs-image.raw.txt new file mode 100644 index 00000000..67f20968 --- /dev/null +++ b/tests/fuzz-tests/images/bko-97271-btrfs-image.raw.txt @@ -0,0 +1,54 @@ +URL: https://bugzilla.kernel.org/show_bug.cgi?id=97271 +Lukas Lueg 2015-04-25 20:34:39 UTC + +The attached btrfs-image causes "btrfs check" to write outside of allocated +memory locations and ultimately die due to a segfault. An adjacent heap block's +control structure is overwritten with a `struct extent_buffer *`, which is not +controllable by the user. + +"btrfs version" is v3.19.1. Running "btrfs check" immediately dies with + +*** Error in `btrfs': double free or corruption (!prev): 0x0000000002396ec0 *** +*** Error in `btrfs': malloc(): memory corruption: 0x0000000002396f60 *** + +Debugging with valgrind and gdb gives + +==11670== Invalid write of size 8 +==11670== at 0x4386FB: btrfs_search_slot (ctree.c:1119) +==11670== by 0x44E16E: btrfs_read_chunk_tree (volumes.c:1814) +==11670== by 0x43D654: btrfs_setup_chunk_tree_and_device_map (disk-io.c:1115) +==11670== by 0x43D7D0: __open_ctree_fd (disk-io.c:1190) +==11670== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==11670== by 0x427BF4: cmd_check (cmds-check.c:9326) +==11670== by 0x40E5A1: main (btrfs.c:245) +==11670== Address 0x4c3bb98 is 8 bytes after a block of size 144 alloc'd +==11670== at 0x4A08946: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) +==11670== by 0x44E133: btrfs_read_chunk_tree (volumes.c:1801) +==11670== by 0x43D654: btrfs_setup_chunk_tree_and_device_map (disk-io.c:1115) +==11670== by 0x43D7D0: __open_ctree_fd (disk-io.c:1190) +==11670== by 0x43D964: open_ctree_fs_info (disk-io.c:1231) +==11670== by 0x427BF4: cmd_check (cmds-check.c:9326) +==11670== by 0x40E5A1: main (btrfs.c:245) + +Program received signal SIGTRAP, Trace/breakpoint trap. +btrfs_search_slot (trans=trans@entry=0x0, root=root@entry=0x4c36d30, key=key@entry=0xffefff830, p=p@entry=0x4c3bb00, + ins_len=ins_len@entry=0, cow=cow@entry=0) at ctree.c:1119 +1119 p->nodes[level] = b; +(gdb) p p->nodes +$1 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0} +(gdb) p p +$2 = (struct btrfs_path *) 0x4c3bb00 +(gdb) p b +$3 = (struct extent_buffer *) 0x4c3a990 + + +The corresponding part in ctree.c:btrfs_search_slot() seems to fail to check if `level` overflows outside of `node`: + +level = btrfs_header_level(b); +... +if (level != btrfs_header_level(b)) + WARN_ON(1); +level = btrfs_header_level(b); +p->nodes[level] = b; // <- Illegal write + +Maybe the repeated calls to btrfs_header_level() were meant to do something once, they seem to be noise. diff --git a/tests/fuzz-tests/images/bko-97271-btrfs-image.raw.xz b/tests/fuzz-tests/images/bko-97271-btrfs-image.raw.xz Binary files differnew file mode 100644 index 00000000..3c79edb5 --- /dev/null +++ b/tests/fuzz-tests/images/bko-97271-btrfs-image.raw.xz diff --git a/tests/fuzz-tests/images/superblock-stripsize-bogus.raw.txt b/tests/fuzz-tests/images/superblock-stripsize-bogus.raw.txt new file mode 100644 index 00000000..80e073f6 --- /dev/null +++ b/tests/fuzz-tests/images/superblock-stripsize-bogus.raw.txt @@ -0,0 +1,32 @@ +[ 125.415910] BTRFS info (device loop0): disk space caching is enabled +[ 125.550479] ------------[ cut here ]------------ +[ 125.551145] WARNING: CPU: 6 PID: 1496 at fs/btrfs/locking.c:251 btrfs_tree_lock+0x22e/0x250 +[ 125.552292] Modules linked in: +[ 125.552602] CPU: 6 PID: 1496 Comm: btrfs.exe Tainted: G W 4.6.0-rc5 #130 +[ 125.553138] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014 +[ 125.553775] 0000000000000286 000000009b4bdd50 ffff88006a7478e0 ffffffff8157e563 +[ 125.554299] 0000000000000000 0000000000000000 ffff88006a747920 ffffffff810a74ab +[ 125.554825] 000000fb8146c531 ffff88006bfec460 ffff88006bc63000 0000000000000000 +[ 125.555373] Call Trace: +[ 125.555545] [<ffffffff8157e563>] dump_stack+0x85/0xc2 +[ 125.555892] [<ffffffff810a74ab>] __warn+0xcb/0xf0 +[ 125.556226] [<ffffffff810a75dd>] warn_slowpath_null+0x1d/0x20 +[ 125.556654] [<ffffffff814871ee>] btrfs_tree_lock+0x22e/0x250 +[ 125.557041] [<ffffffff81423831>] btrfs_init_new_buffer+0x81/0x160 +[ 125.557458] [<ffffffff8143472a>] btrfs_alloc_tree_block+0x22a/0x430 +[ 125.557883] [<ffffffff8141ae61>] __btrfs_cow_block+0x141/0x590 +[ 125.558279] [<ffffffff8141b44f>] btrfs_cow_block+0x11f/0x1f0 +[ 125.558666] [<ffffffff8141f09e>] btrfs_search_slot+0x1fe/0xa30 +[ 125.559063] [<ffffffff81247c9d>] ? kmem_cache_alloc+0xfd/0x240 +[ 125.559482] [<ffffffff8143b1f0>] btrfs_del_inode_ref+0x80/0x380 +[ 125.559884] [<ffffffff8148e11a>] ? btrfs_del_inode_ref_in_log+0x8a/0x160 +[ 125.560340] [<ffffffff8148e14d>] btrfs_del_inode_ref_in_log+0xbd/0x160 +[ 125.560776] [<ffffffff814507f7>] __btrfs_unlink_inode+0x1d7/0x470 +[ 125.561188] [<ffffffff814567a7>] btrfs_rename2+0x327/0x790 +[ 125.561568] [<ffffffff8127b398>] vfs_rename+0x4d8/0x840 +[ 125.561928] [<ffffffff81281b21>] SyS_rename+0x371/0x390 +[ 125.562289] [<ffffffff819cfd3c>] entry_SYSCALL_64_fastpath+0x1f/0xbd +[ 125.562743] ---[ end trace 3b751f511705fb90 ]--- + +--------------------------------------------------------------------------- +Fixed by patch: diff --git a/tests/fuzz-tests/images/superblock-stripsize-bogus.raw.xz b/tests/fuzz-tests/images/superblock-stripsize-bogus.raw.xz Binary files differnew file mode 100644 index 00000000..f8b3bf54 --- /dev/null +++ b/tests/fuzz-tests/images/superblock-stripsize-bogus.raw.xz diff --git a/tests/fuzz-tests/images/superblock-total-bytes-0.raw.txt b/tests/fuzz-tests/images/superblock-total-bytes-0.raw.txt new file mode 100644 index 00000000..d5e1f936 --- /dev/null +++ b/tests/fuzz-tests/images/superblock-total-bytes-0.raw.txt @@ -0,0 +1,50 @@ +[342246.846031] BTRFS info (device loop0): disk space caching is enabled +[342246.862115] ------------[ cut here ]------------ +[342246.862500] kernel BUG at fs/btrfs/inode.c:978! +[342246.862861] invalid opcode: 0000 [#1] SMP +[342246.863176] Modules linked in: +[342246.863410] CPU: 2 PID: 14504 Comm: btrfs.exe Tainted: G W 4.6.0-rc5 #130 +[342246.864010] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014 +[342246.864674] task: ffff88006fdf0000 ti: ffff8800702e0000 task.ti: ffff8800702e0000 +[342246.865186] RIP: 0010:[<ffffffff8144e9c7>] [<ffffffff8144e9c7>] cow_file_range+0x3f7/0x440 +[342246.865770] RSP: 0018:ffff8800702e39e0 EFLAGS: 00010206 +[342246.866157] RAX: ffff88006bb23000 RBX: 0000000000000001 RCX: 0000000000010000 +[342246.866687] RDX: 0000000000000000 RSI: 0000000000001000 RDI: 0000000000010000 +[342246.867191] RBP: ffff8800702e3a70 R08: 0000000000000000 R09: 0000000000000000 +[342246.867682] R10: 000000000000ffff R11: 0000000000010000 R12: ffff8800702e3bc0 +[342246.868170] R13: ffff8800702e3b3c R14: 0000000000000000 R15: ffff880075369c10 +[342246.868660] FS: 00007f96f5a38700(0000) GS:ffff88007ca00000(0000) knlGS:0000000000000000 +[342246.869212] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 +[342246.869642] CR2: 000000000060f4bf CR3: 000000006fc9f000 CR4: 00000000000006e0 +[342246.870146] Stack: +[342246.870295] 0000000000000000 0000000000000001 000000000000ffff ffffea00010c08c0 +[342246.870838] ffff8800753698e8 0000000000010000 ffff88006fe0f000 000000000000ffff +[342246.871397] 000000000000ffff ffffffff814683e5 ffff8800753698c8 ffff8800753698e8 +[342246.871944] Call Trace: +[342246.872124] [<ffffffff814683e5>] ? test_range_bit+0xe5/0x130 +[342246.872522] [<ffffffff8144f906>] run_delalloc_range+0x396/0x3d0 +[342246.872975] [<ffffffff8146873f>] writepage_delalloc.isra.42+0x10f/0x170 +[342246.873437] [<ffffffff8146a674>] __extent_writepage+0xf4/0x370 +[342246.873848] [<ffffffff8146abf4>] extent_write_cache_pages.isra.39.constprop.57+0x304/0x3f0 +[342246.874419] [<ffffffff8146beec>] extent_writepages+0x5c/0x90 +[342246.874818] [<ffffffff8144c870>] ? btrfs_real_readdir+0x5f0/0x5f0 +[342246.875245] [<ffffffff814498f8>] btrfs_writepages+0x28/0x30 +[342246.875641] [<ffffffff811ebc61>] do_writepages+0x21/0x30 +[342246.876031] [<ffffffff811dc1a6>] __filemap_fdatawrite_range+0xc6/0x100 +[342246.876487] [<ffffffff811dc2b3>] filemap_fdatawrite_range+0x13/0x20 +[342246.876949] [<ffffffff8145eae0>] btrfs_fdatawrite_range+0x20/0x50 +[342246.877375] [<ffffffff8145eb29>] start_ordered_ops+0x19/0x30 +[342246.877774] [<ffffffff8145ebc2>] btrfs_sync_file+0x82/0x3f0 +[342246.878166] [<ffffffff810fb717>] ? update_fast_ctr+0x17/0x30 +[342246.878564] [<ffffffff812a848b>] vfs_fsync_range+0x4b/0xb0 +[342246.878987] [<ffffffff8128fce6>] ? __fget_light+0x66/0x90 +[342246.879368] [<ffffffff812a854d>] do_fsync+0x3d/0x70 +[342246.879708] [<ffffffff812a8823>] SyS_fdatasync+0x13/0x20 +[342246.880099] [<ffffffff819cfd3c>] entry_SYSCALL_64_fastpath+0x1f/0xbd +[342246.880554] Code: 03 00 00 48 c7 c7 00 b3 c9 81 c6 05 54 b6 b1 00 01 e8 0e 8c c5 ff e9 e5 fe ff ff 49 8b 57 40 e9 c0 fe ff ff bb f4 ff ff ff eb a1 <0f> 0b 48 8b 55 80 41 b9 0f 00 00 00 41 b8 68 00 00 00 31 c9 31 +[342246.882394] RIP [<ffffffff8144e9c7>] cow_file_range+0x3f7/0x440 +[342246.882810] RSP <ffff8800702e39e0> +[342246.883076] ---[ end trace 094193b6df6e45e7 ]--- + +-------------------------------------------------------- +Fixed by patch: diff --git a/tests/fuzz-tests/images/superblock-total-bytes-0.raw.xz b/tests/fuzz-tests/images/superblock-total-bytes-0.raw.xz Binary files differnew file mode 100644 index 00000000..4b25020e --- /dev/null +++ b/tests/fuzz-tests/images/superblock-total-bytes-0.raw.xz diff --git a/tests/fuzz-tests/images/sys-array-num-stripes-0.raw.txt b/tests/fuzz-tests/images/sys-array-num-stripes-0.raw.txt new file mode 100644 index 00000000..bdde4e70 --- /dev/null +++ b/tests/fuzz-tests/images/sys-array-num-stripes-0.raw.txt @@ -0,0 +1,30 @@ +URL: http://article.gmane.org/gmane.comp.file-systems.btrfs/50230 +Vegard Nossum, 2015-11-15 + +If sys_array::num_stripes == 0, we hit a BUG_ON during mount: + +BTRFS: device fsid 9006933e-2a9a-44f0-917f-514252aeec2c devid 1 transid 7 /dev/loop0 +BTRFS info (device loop0): disk space caching is enabled +BUG: failure at fs/btrfs/ctree.h:337/btrfs_chunk_item_size()! +Kernel panic - not syncing: BUG! +CPU: 0 PID: 313 Comm: mount Not tainted 4.2.5-00657-ge047887-dirty #25 +Stack: + 637af890 60062489 602aeb2e 604192ba + 60387961 00000011 637af8a0 6038a835 + 637af9c0 6038776b 634ef32b 00000000 +Call Trace: + [<6001c86d>] show_stack+0xfe/0x15b + [<6038a835>] dump_stack+0x2a/0x2c + [<6038776b>] panic+0x13e/0x2b3 + [<6020f099>] btrfs_read_sys_array+0x25d/0x2ff + [<601cfbbe>] open_ctree+0x192d/0x27af + [<6019c2c1>] btrfs_mount+0x8f5/0xb9a + [<600bc9a7>] mount_fs+0x11/0xf3 + [<600d5167>] vfs_kern_mount+0x75/0x11a + [<6019bcb0>] btrfs_mount+0x2e4/0xb9a + [<600bc9a7>] mount_fs+0x11/0xf3 + [<600d5167>] vfs_kern_mount+0x75/0x11a + [<600d710b>] do_mount+0xa35/0xbc9 + [<600d7557>] SyS_mount+0x95/0xc8 + +Fixed by patch (kernel and btrfs-progs): btrfs: handle invalid num_stripes in sys_array diff --git a/tests/fuzz-tests/images/sys-array-num-stripes-0.raw.xz b/tests/fuzz-tests/images/sys-array-num-stripes-0.raw.xz Binary files differnew file mode 100644 index 00000000..d64fb300 --- /dev/null +++ b/tests/fuzz-tests/images/sys-array-num-stripes-0.raw.xz diff --git a/tests/fuzz-tests/images/sys-chunk-stripe-len-bogus.raw.txt b/tests/fuzz-tests/images/sys-chunk-stripe-len-bogus.raw.txt new file mode 100644 index 00000000..d3dcb0a4 --- /dev/null +++ b/tests/fuzz-tests/images/sys-chunk-stripe-len-bogus.raw.txt @@ -0,0 +1,54 @@ +[ 135.166891] BTRFS info (device loop0): disk space caching is enabled +[ 135.169199] divide error: 0000 [#1] SMP +[ 135.169581] Modules linked in: +[ 135.169819] CPU: 2 PID: 1512 Comm: btrfs.exe Tainted: G W 4.6.0-rc5 #130 +[ 135.170285] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014 +[ 135.170958] task: ffff880074925180 ti: ffff880077fa4000 task.ti: ffff880077fa4000 +[ 135.171583] RIP: 0010:[<ffffffff81475ba0>] [<ffffffff81475ba0>] __btrfs_map_block+0xc0/0x11b0 +[ 135.172096] RSP: 0000:ffff880077fa77b0 EFLAGS: 00010206 +[ 135.172374] RAX: 0000000000020000 RBX: 0000000000020000 RCX: 0000000000000000 +[ 135.172754] RDX: 0000000000000000 RSI: 0000000000400000 RDI: ffff880076258270 +[ 135.173143] RBP: ffff880077fa7898 R08: 0000000000400000 R09: 0000000000000000 +[ 135.173523] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000020000 +[ 135.173916] R13: ffff880076258270 R14: ffff880077fa78e0 R15: ffff88006bb3b000 +[ 135.174290] FS: 00007fd8267dc700(0000) GS:ffff88007ca00000(0000) knlGS:0000000000000000 +[ 135.174718] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 +[ 135.175019] CR2: 00007ffe9c378df7 CR3: 0000000078788000 CR4: 00000000000006e0 +[ 135.175392] Stack: +[ 135.175503] ffff88007cbe2c40 0000000000000000 ffff88007cbe2c50 ffff880074925180 +[ 135.175924] ffff880074926560 ffff880074925180 0000000200000000 0000000000000000 +[ 135.176340] ffffffffffffffff 0007ffffffffffff ffffffff8143eb18 0240004000000000 +[ 135.176778] Call Trace: +[ 135.176913] [<ffffffff8143eb18>] ? btrfs_bio_wq_end_io+0x28/0x70 +[ 135.177234] [<ffffffff81477218>] btrfs_map_bio+0x88/0x350 +[ 135.177522] [<ffffffff8143eb18>] ? btrfs_bio_wq_end_io+0x28/0x70 +[ 135.177960] [<ffffffff8143ed9d>] btree_submit_bio_hook+0x6d/0x110 +[ 135.178410] [<ffffffff81464d1d>] submit_one_bio+0x6d/0xa0 +[ 135.178814] [<ffffffff8146d6f1>] read_extent_buffer_pages+0x1c1/0x350 +[ 135.179276] [<ffffffff8143cd60>] ? free_root_pointers+0x70/0x70 +[ 135.179708] [<ffffffff8143e12c>] btree_read_extent_buffer_pages.constprop.55+0xac/0x110 +[ 135.180261] [<ffffffff8143f036>] read_tree_block+0x36/0x60 +[ 135.180647] [<ffffffff81443b52>] open_ctree+0x17a2/0x2900 +[ 135.181027] [<ffffffff81417225>] btrfs_mount+0xd05/0xe60 +[ 135.181400] [<ffffffff819cd15a>] ? __mutex_unlock_slowpath+0xfa/0x1c0 +[ 135.181850] [<ffffffff810fd3e4>] ? lockdep_init_map+0x64/0x710 +[ 135.182241] [<ffffffff81272918>] mount_fs+0x38/0x170 +[ 135.182609] [<ffffffff81292b7b>] vfs_kern_mount+0x6b/0x150 +[ 135.182998] [<ffffffff814166e6>] btrfs_mount+0x1c6/0xe60 +[ 135.183372] [<ffffffff819cd15a>] ? __mutex_unlock_slowpath+0xfa/0x1c0 +[ 135.183825] [<ffffffff810fd3e4>] ? lockdep_init_map+0x64/0x710 +[ 135.184233] [<ffffffff81272918>] mount_fs+0x38/0x170 +[ 135.184583] [<ffffffff81292b7b>] vfs_kern_mount+0x6b/0x150 +[ 135.184971] [<ffffffff812958c6>] do_mount+0x256/0xeb0 +[ 135.185318] [<ffffffff8124bb33>] ? __kmalloc_track_caller+0x113/0x290 +[ 135.185759] [<ffffffff812b0b63>] ? block_ioctl+0x43/0x50 +[ 135.186124] [<ffffffff811ff023>] ? memdup_user+0x53/0x80 +[ 135.186488] [<ffffffff81296865>] SyS_mount+0x95/0xe0 +[ 135.186877] [<ffffffff819cfd3c>] entry_SYSCALL_64_fastpath+0x1f/0xbd +[ 135.187308] Code: 8b 70 20 4c 8d 04 31 4c 39 c3 0f 87 2f 0b 00 00 48 8b 45 a8 49 89 dc 31 d2 49 29 cc 48 8b 40 70 48 63 48 10 48 89 45 a0 4c 89 e0 <48> f7 f1 49 89 cf 48 89 45 b8 48 0f af c1 49 39 c4 0f 82 c3 0a +[ 135.189097] RIP [<ffffffff81475ba0>] __btrfs_map_block+0xc0/0x11b0 +[ 135.189527] RSP <ffff880077fa77b0> +[ 135.189819] ---[ end trace ea21fae64670799a ]--- + +--------------------------------------------------------------------------- +Fixed by patch: diff --git a/tests/fuzz-tests/images/sys-chunk-stripe-len-bogus.raw.xz b/tests/fuzz-tests/images/sys-chunk-stripe-len-bogus.raw.xz Binary files differnew file mode 100644 index 00000000..57d2a72f --- /dev/null +++ b/tests/fuzz-tests/images/sys-chunk-stripe-len-bogus.raw.xz diff --git a/tests/fuzz-tests/images/sys-chunk-type-bogus.raw.txt b/tests/fuzz-tests/images/sys-chunk-type-bogus.raw.txt new file mode 100644 index 00000000..2559924d --- /dev/null +++ b/tests/fuzz-tests/images/sys-chunk-type-bogus.raw.txt @@ -0,0 +1,55 @@ +[ 145.676440] BTRFS error (device loop0): bad tree block start 0 131072 +[ 145.677032] ------------[ cut here ]------------ +[ 145.677307] kernel BUG at fs/btrfs/raid56.c:2142! +[ 145.677627] invalid opcode: 0000 [#1] SMP +[ 145.677955] Modules linked in: +[ 145.678182] CPU: 3 PID: 1538 Comm: btrfs.exe Tainted: G W 4.6.0-rc5 #130 +[ 145.678734] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014 +[ 145.679402] task: ffff88006c830000 ti: ffff88006fc74000 task.ti: ffff88006fc74000 +[ 145.679919] RIP: 0010:[<ffffffff814c5794>] [<ffffffff814c5794>] raid56_parity_recover+0xc4/0x160 +[ 145.680514] RSP: 0018:ffff88006fc77868 EFLAGS: 00010286 +[ 145.680865] RAX: ffff88006f725280 RBX: ffff880070ba0a68 RCX: 0000000000020000 +[ 145.681373] RDX: 0000000000000100 RSI: 00000000ffffffff RDI: ffffffff831229e8 +[ 145.681866] RBP: ffff88006fc77898 R08: 0000000000010000 R09: ffff8800768ff400 +[ 145.682380] R10: ffff88007c003180 R11: 0000000000030000 R12: ffff88006f725280 +[ 145.682870] R13: ffff88007b449000 R14: 0000000000000001 R15: ffff8800768ff400 +[ 145.683363] FS: 00007f68b95a8700(0000) GS:ffff88007cc00000(0000) knlGS:0000000000000000 +[ 145.683941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 +[ 145.684340] CR2: 00007fff0d130f98 CR3: 000000006bfd7000 CR4: 00000000000006e0 +[ 145.684832] Stack: +[ 145.684977] 00000002e6816dd1 ffff880070ba0a68 ffff88007b449000 0000000000000001 +[ 145.685541] 0000000000020000 0000000000000002 ffff88006fc77920 ffffffff814773cd +[ 145.686082] ffff880000000001 0000000002400040 ffff88006fc778f8 0000000081247c9d +[ 145.686654] Call Trace: +[ 145.686831] [<ffffffff814773cd>] btrfs_map_bio+0x23d/0x350 +[ 145.687217] [<ffffffff8143ed9d>] btree_submit_bio_hook+0x6d/0x110 +[ 145.687649] [<ffffffff81464d1d>] submit_one_bio+0x6d/0xa0 +[ 145.688028] [<ffffffff8146d6f1>] read_extent_buffer_pages+0x1c1/0x350 +[ 145.688501] [<ffffffff8143cd60>] ? free_root_pointers+0x70/0x70 +[ 145.688916] [<ffffffff8143e12c>] btree_read_extent_buffer_pages.constprop.55+0xac/0x110 +[ 145.689474] [<ffffffff8143f036>] read_tree_block+0x36/0x60 +[ 145.689861] [<ffffffff81443b52>] open_ctree+0x17a2/0x2900 +[ 145.690242] [<ffffffff81417225>] btrfs_mount+0xd05/0xe60 +[ 145.690623] [<ffffffff819cd15a>] ? __mutex_unlock_slowpath+0xfa/0x1c0 +[ 145.691064] [<ffffffff810fd3e4>] ? lockdep_init_map+0x64/0x710 +[ 145.691510] [<ffffffff81272918>] mount_fs+0x38/0x170 +[ 145.691852] [<ffffffff81292b7b>] vfs_kern_mount+0x6b/0x150 +[ 145.692227] [<ffffffff814166e6>] btrfs_mount+0x1c6/0xe60 +[ 145.692594] [<ffffffff819cd15a>] ? __mutex_unlock_slowpath+0xfa/0x1c0 +[ 145.693032] [<ffffffff810fd3e4>] ? lockdep_init_map+0x64/0x710 +[ 145.693453] [<ffffffff81272918>] mount_fs+0x38/0x170 +[ 145.693793] [<ffffffff81292b7b>] vfs_kern_mount+0x6b/0x150 +[ 145.694168] [<ffffffff812958c6>] do_mount+0x256/0xeb0 +[ 145.694537] [<ffffffff8124bb33>] ? __kmalloc_track_caller+0x113/0x290 +[ 145.694974] [<ffffffff812b0b63>] ? block_ioctl+0x43/0x50 +[ 145.695338] [<ffffffff811ff023>] ? memdup_user+0x53/0x80 +[ 145.695703] [<ffffffff81296865>] SyS_mount+0x95/0xe0 +[ 145.696046] [<ffffffff819cfd3c>] entry_SYSCALL_64_fastpath+0x1f/0xbd +[ 145.696480] Code: 1f 48 8b 78 58 31 c0 48 8b 14 c7 48 39 d1 72 08 4c 01 c2 48 39 d1 72 15 48 83 c0 01 39 c6 7f e7 41 c7 87 3c 01 00 00 ff ff ff ff <0f> 0b 45 85 f6 41 89 87 3c 01 00 00 75 35 4c 89 e7 e8 e6 02 fb +[ 145.698326] RIP [<ffffffff814c5794>] raid56_parity_recover+0xc4/0x160 +[ 145.698771] RSP <ffff88006fc77868> +[ 145.699047] ---[ end trace 22f39f01df276367 ]--- + +----------------------------------------------------- +Fixed by patch: + diff --git a/tests/fuzz-tests/images/sys-chunk-type-bogus.raw.xz b/tests/fuzz-tests/images/sys-chunk-type-bogus.raw.xz Binary files differnew file mode 100644 index 00000000..ef971ca3 --- /dev/null +++ b/tests/fuzz-tests/images/sys-chunk-type-bogus.raw.xz diff --git a/tests/misc-tests.sh b/tests/misc-tests.sh new file mode 100755 index 00000000..eefe8a81 --- /dev/null +++ b/tests/misc-tests.sh @@ -0,0 +1,43 @@ +#!/bin/bash +# +# Misc tests + +LANG=C +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +TEST_DEV=${TEST_DEV:-} +RESULTS="$TOP/tests/misc-tests-results.txt" +IMAGE="$TOP/tests/test.img" + +source $TOP/tests/common + +export TOP +export RESULTS +export LANG +export TEST_DEV +export IMAGE + +rm -f $RESULTS + +# test rely on corrupting blocks tool +check_prereq btrfs-corrupt-block +check_prereq btrfs-image +check_prereq btrfstune +check_prereq btrfs + +# The tests are driven by their custom script called 'test.sh' + +for i in $(find $TOP/tests/misc-tests -maxdepth 1 -mindepth 1 -type d \ + ${TEST:+-name "$TEST"} | sort) +do + echo " [TEST/misc] $(basename $i)" + cd $i + echo "=== Entering $i" >> $RESULTS + if [ -x test.sh ]; then + ./test.sh + if [ $? -ne 0 ]; then + _fail "test failed for case $(basename $i)" + fi + fi + cd $TOP +done diff --git a/tests/misc-tests/001-btrfstune-features/test.sh b/tests/misc-tests/001-btrfstune-features/test.sh new file mode 100755 index 00000000..c858d701 --- /dev/null +++ b/tests/misc-tests/001-btrfstune-features/test.sh @@ -0,0 +1,46 @@ +#!/bin/bash +# test btrfstune options that enable filesystem features + +source $TOP/tests/common + +check_prereq btrfs-debug-tree +check_prereq btrfs-show-super +check_prereq mkfs.btrfs +check_prereq btrfstune +check_prereq btrfs + +setup_root_helper +prepare_test_dev + +# test whether fsck can rebuild a corrupted extent tree +# parameters: +# - option for mkfs.btrfs -O, empty for defaults +# - option for btrfstune +# - string representing the feature in btrfs-show-super dump +test_feature() +{ + local mkfsfeatures + local tuneopt + local sbflag + + mkfsfeatures=${1:+-O ^$1} + tuneopt="$2" + sbflag="$3" + + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f $mkfsfeatures $TEST_DEV + if run_check_stdout $TOP/btrfs-show-super $TEST_DEV | \ + grep -q "$sbflag"; then + _fail "FAIL: feature $sbflag must not be set on the base image" + fi + run_check $TOP/btrfstune $tuneopt $TEST_DEV + if ! run_check_stdout $TOP/btrfs-show-super $TEST_DEV | \ + grep -q "$sbflag"; then + _fail "FAIL: feature $sbflag not set" + fi + run_check $SUDO_HELPER $TOP/btrfs check $TEST_DEV +} + +test_feature extref -r EXTENDED_IREF +test_feature skinny-metadata -x SKINNY_METADATA +test_feature no-holes -n NO_HOLES +test_feature '' '-S 1' SEEDING diff --git a/tests/misc-tests/002-uuid-rewrite/test.sh b/tests/misc-tests/002-uuid-rewrite/test.sh new file mode 100755 index 00000000..d84ec6ca --- /dev/null +++ b/tests/misc-tests/002-uuid-rewrite/test.sh @@ -0,0 +1,66 @@ +#!/bin/bash +# test btrfstune uuid rewriting options + +source $TOP/tests/common + +check_prereq btrfs-debug-tree +check_prereq btrfs-show-super +check_prereq mkfs.btrfs +check_prereq btrfstune +check_prereq btrfs + +prepare_test_dev + +get_fs_uuid() { + local image + + image="$1" + run_check_stdout $TOP/btrfs-show-super "$image" | \ + grep '^fsid' | awk '{print $2}' +} + +test_uuid_random() +{ + local origuuid + + origuuid=11111111-a101-4031-b29a-379d4f8b7a2d + + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f \ + --uuid $origuuid \ + --rootdir $TOP/Documentation \ + $TEST_DEV + run_check $TOP/btrfs-show-super "$TEST_DEV" + currentfsid=$(run_check_stdout $TOP/btrfstune -f -u $TEST_DEV | \ + grep -i 'current fsid:' | awk '{print $3}') + if ! [ $currentfsid = $origuuid ]; then + _fail "FAIL: current UUID mismatch" + fi + run_check $TOP/btrfs-show-super "$TEST_DEV" + run_check $SUDO_HELPER $TOP/btrfs check $TEST_DEV +} + +test_uuid_user() +{ + local origuuid + local newuuid + + origuuid=22222222-d324-4f92-80e9-7658bf3b845f + newuuid=33333333-bfc9-4045-9399-a396dc6893b3 + + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f \ + --uuid $origuuid \ + --rootdir $TOP/Documentation \ + $TEST_DEV + run_check $TOP/btrfs-show-super "$TEST_DEV" + run_check $TOP/btrfstune -f -U $newuuid \ + $TEST_DEV + # btrfs-show-super is called within get_fs_uuid + fsid=$(get_fs_uuid $TEST_DEV) + if ! [ $fsid = $newuuid ]; then + _fail "FAIL: UUID not rewritten" + fi + run_check $SUDO_HELPER $TOP/btrfs check $TEST_DEV +} + +test_uuid_random +test_uuid_user diff --git a/tests/misc-tests/003-zero-log/test.sh b/tests/misc-tests/003-zero-log/test.sh new file mode 100755 index 00000000..b650930e --- /dev/null +++ b/tests/misc-tests/003-zero-log/test.sh @@ -0,0 +1,52 @@ +#!/bin/bash +# test zero-log + +source $TOP/tests/common + +check_prereq btrfs-show-super +check_prereq mkfs.btrfs +check_prereq btrfs +prepare_test_dev + +get_log_root() +{ + local image + + image="$1" + $TOP/btrfs-show-super "$image" | \ + grep '^log_root\>' | awk '{print $2}' +} +get_log_root_level() { + local image + + image="$1" + $TOP/btrfs-show-super "$image" | \ + grep '^log_root_level' | awk '{print $2}' +} + +test_zero_log() +{ + # FIXME: we need an image with existing log_root + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f \ + --rootdir $TOP/Documentation \ + $TEST_DEV + run_check $TOP/btrfs-show-super $TEST_DEV + if [ "$1" = 'standalone' ]; then + run_check $TOP/btrfs rescue zero-log $TEST_DEV + else + run_check $TOP/btrfs-zero-log $TEST_DEV + fi + log_root=$(get_log_root $TEST_DEV) + log_root_level=$(get_log_root $TEST_DEV) + if [ "$log_root" != 0 ]; then + _fail "FAIL: log_root not reset" + fi + if [ "$log_root_level" != 0 ]; then + _fail "FAIL: log_root_level not reset" + fi + run_check $TOP/btrfs-show-super $TEST_DEV + run_check $SUDO_HELPER $TOP/btrfs check $TEST_DEV +} + +test_zero_log standalone +test_zero_log internal diff --git a/tests/misc-tests/004-shrink-fs/test.sh b/tests/misc-tests/004-shrink-fs/test.sh new file mode 100755 index 00000000..88740358 --- /dev/null +++ b/tests/misc-tests/004-shrink-fs/test.sh @@ -0,0 +1,72 @@ +#!/bin/bash +# +# Test getting the minimum size a filesystem can be resized to and verify we +# are able to resize (shrink) it to that size. +# + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper + +# Optionally take id of the device to shrink +shrink_test() +{ + min_size=$(run_check_stdout $SUDO_HELPER $TOP/btrfs inspect-internal min-dev-size ${1:+--id $1} $TEST_MNT) + min_size=$(echo $min_size | cut -d ' ' -f 1) + echo "min size = ${min_size}" >> $RESULTS + if [ -z "$min_size" ]; then + _fail "Failed to parse minimum size" + fi + run_check $SUDO_HELPER $TOP/btrfs filesystem resize $min_size $TEST_MNT +} + +run_check truncate -s 20G $IMAGE +run_check $TOP/mkfs.btrfs -f $IMAGE +run_check $SUDO_HELPER mount $IMAGE $TEST_MNT +run_check $SUDO_HELPER chmod a+rw $TEST_MNT + +# Create 7 data block groups, each with a size of 1Gb. +for ((i = 1; i <= 7; i++)); do + run_check fallocate -l 1G $TEST_MNT/foo$i +done + +# Make sure they are persisted (all the chunk, device and block group items +# added to the chunk/dev/extent trees). +run_check $TOP/btrfs filesystem sync $TEST_MNT + +# Now remove 3 of those 1G files. This will result in 3 block groups becoming +# unused, which will be automatically deleted by the cleaner kthread, and this +# will result in 3 holes (unallocated space) in the device (each with a size +# of 1Gb). + +run_check rm -f $TEST_MNT/foo2 +run_check rm -f $TEST_MNT/foo4 +run_check rm -f $TEST_MNT/foo6 + +# Sync once to wake up the cleaner kthread which will delete the unused block +# groups - it could have been sleeping when they became unused. Then wait a bit +# to allow the cleaner kthread to delete them and then finally ensure the +# transaction started by the cleaner kthread is committed. +run_check $TOP/btrfs filesystem sync $TEST_MNT +sleep 3 +run_check $TOP/btrfs filesystem sync $TEST_MNT + +# Now attempt to get the minimum size we can resize the filesystem to and verify +# the resize operation succeeds. This size closely matches the sum of the size +# of all the allocated device extents. +for ((i = 1; i <= 3; i++)); do + shrink_test +done + +# Now convert metadata and system chunks to the single profile and check we are +# still able to get a correct minimum size and shrink to that size. +run_check $SUDO_HELPER $TOP/btrfs balance start -mconvert=single \ + -sconvert=single -f $TEST_MNT +for ((i = 1; i <= 3; i++)); do + shrink_test 1 +done + +run_check $SUDO_HELPER umount $TEST_MNT diff --git a/tests/misc-tests/005-convert-progress-thread-crash/test.sh b/tests/misc-tests/005-convert-progress-thread-crash/test.sh new file mode 100755 index 00000000..054069c2 --- /dev/null +++ b/tests/misc-tests/005-convert-progress-thread-crash/test.sh @@ -0,0 +1,15 @@ +#!/bin/bash +# test convert-thread-conflict + +source $TOP/tests/common + +check_prereq btrfs-convert + +mkfs.ext4 -V &>/dev/null || _not_run "mkfs.ext4 not found" +prepare_test_dev 1G + +for ((i = 0; i < 20; i++)); do + echo "loop $i" >>$RESULTS + mkfs.ext4 -F "$TEST_DEV" &>>$RESULTS || _not_run "mkfs.ext4 failed" + run_check $TOP/btrfs-convert "$TEST_DEV" +done diff --git a/tests/misc-tests/006-image-on-missing-device/test.sh b/tests/misc-tests/006-image-on-missing-device/test.sh new file mode 100755 index 00000000..8680a707 --- /dev/null +++ b/tests/misc-tests/006-image-on-missing-device/test.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# test btrfs-image with a missing device (uses loop devices) +# +# - btrfs-image must not loop indefinetelly +# - btrfs-image will expectedly fail to produce the dump + +source $TOP/tests/common + +check_prereq btrfs-show-super +check_prereq btrfs-image +check_prereq mkfs.btrfs +check_prereq btrfs + +ndevs=2 +declare -a devs +dev1= +dev2= + +setup_root_helper + + +# TODO: move the helpers to common + +prepare_devices() +{ + for i in `seq $ndevs`; do + touch img$i + chmod a+rw img$i + truncate -s0 img$i + truncate -s2g img$i + devs[$i]=`run_check_stdout $SUDO_HELPER losetup --find --show img$i` + done +} + +cleanup_devices() +{ + for dev in ${devs[@]}; do + run_mayfail $SUDO_HELPER losetup -d $dev + done + for i in `seq $ndevs`; do + truncate -s0 img$i + done + run_check $SUDO_HELPER losetup --list +} + +test_image_dump() +{ + run_check $SUDO_HELPER $TOP/btrfs check $dev1 + # the output file will be deleted + run_mayfail $SUDO_HELPER $TOP/btrfs-image $dev1 /tmp/test-img.dump +} + +test_run() +{ + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f -d raid1 -m raid1 $dev1 $dev2 + + # we need extents to trigger reading from all devices + run_check $SUDO_HELPER mount $dev1 $TEST_MNT + run_check $SUDO_HELPER dd if=/dev/zero of=$TEST_MNT/a bs=1M count=10 + run_check $SUDO_HELPER dd if=/dev/zero of=$TEST_MNT/b bs=4k count=1000 conv=sync + run_check $SUDO_HELPER umount $TEST_MNT + + test_image_dump + run_check btrfs fi show $dev1 + # create a degraded raid1 filesystem, check must succeed + # btrfs-image must not loop + run_mayfail wipefs -a $dev2 + run_check $SUDO_HELPER losetup -d $dev2 + run_check btrfs fi show $dev1 + + test_image_dump +} + +prepare_devices +dev1=${devs[1]} +dev2=${devs[2]} +test_run +cleanup_devices diff --git a/tests/misc-tests/007-subvolume-sync/test.sh b/tests/misc-tests/007-subvolume-sync/test.sh new file mode 100755 index 00000000..243bb8cc --- /dev/null +++ b/tests/misc-tests/007-subvolume-sync/test.sh @@ -0,0 +1,32 @@ +#!/bin/bash +# test btrfs subvolume run normally with more than one subvolume +# +# - btrfs subvolume must not loop indefinitely +# - btrfs subvolume return 0 in normal case + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper +prepare_test_dev + +run_check $SUDO_HELPER $TOP/mkfs.btrfs -f "$TEST_DEV" +run_check_mount_test_dev + +# to check following thing in both 1 and multiple subvolume case: +# 1: is subvolume sync loop indefinitely +# 2: is return value right +# +run_check $SUDO_HELPER $TOP/btrfs subvolume create "$TEST_MNT"/mysubvol1 +run_check $SUDO_HELPER $TOP/btrfs subvolume create "$TEST_MNT"/mysubvol2 +run_check $SUDO_HELPER $TOP/btrfs subvolume delete "$TEST_MNT"/mysubvol1 +run_check $SUDO_HELPER $TOP/btrfs subvolume delete "$TEST_MNT"/mysubvol2 +run_check $SUDO_HELPER $TOP/btrfs subvolume sync "$TEST_MNT" + +run_check $SUDO_HELPER $TOP/btrfs subvolume create "$TEST_MNT"/mysubvol +run_check $SUDO_HELPER $TOP/btrfs subvolume delete "$TEST_MNT"/mysubvol +run_check $SUDO_HELPER $TOP/btrfs subvolume sync "$TEST_MNT" + +run_check_umount_test_dev diff --git a/tests/misc-tests/008-leaf-crossing-stripes/test.sh b/tests/misc-tests/008-leaf-crossing-stripes/test.sh new file mode 100755 index 00000000..03818062 --- /dev/null +++ b/tests/misc-tests/008-leaf-crossing-stripes/test.sh @@ -0,0 +1,25 @@ +#!/bin/bash +# test if btrfs-convert creates a filesystem without leaf crossing stripes + +source $TOP/tests/common + +check_prereq btrfs-convert +check_prereq btrfs + +# In my test, it happened in 514M~560M, 737M~769M, 929M~917M, +# and HAVE_ERROR=((size + 1) / 2) % 2 if size >= 970 +# +SIZE_FROM=514 +SIZE_END=560 +A_PRIME_NUM=17 +for ((size = SIZE_FROM; size <= SIZE_END; size += A_PRIME_NUM)); do + run_check truncate -s "$size"M "$IMAGE" + run_check mkfs.ext4 -F "$IMAGE" + run_check $TOP/btrfs-convert "$IMAGE" + run_check_stdout $TOP/btrfs check "$IMAGE" 2>&1 | + grep -q "crossing stripe boundary" && + _fail "leaf crossing stripes after btrfs-convert" +done + +# grep will expectedly fail +exit 0 diff --git a/tests/misc-tests/009-subvolume-sync-must-wait/test.sh b/tests/misc-tests/009-subvolume-sync-must-wait/test.sh new file mode 100755 index 00000000..92c896f9 --- /dev/null +++ b/tests/misc-tests/009-subvolume-sync-must-wait/test.sh @@ -0,0 +1,52 @@ +#!/bin/bash +# +# Verify that subvolume sync waits until the subvolume is cleaned + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper + +run_check truncate -s 2G $IMAGE +run_check $TOP/mkfs.btrfs -f $IMAGE +run_check $SUDO_HELPER mount $IMAGE $TEST_MNT +run_check $SUDO_HELPER chmod a+rw $TEST_MNT + +cd $TEST_MNT + +for i in `seq 5`; do + run_check dd if=/dev/zero of=file$i bs=1M count=10 +done + +for sn in `seq 4`;do + run_check $SUDO_HELPER $TOP/btrfs subvolume snapshot . snap$sn + for i in `seq 10`; do + run_check dd if=/dev/zero of=snap$sn/file$i bs=1M count=10 + done +done + +run_check $SUDO_HELPER $TOP/btrfs subvolume list . +run_check $SUDO_HELPER $TOP/btrfs subvolume list -d . + +idtodel=`run_check_stdout $SUDO_HELPER $TOP/btrfs inspect-internal rootid snap3` + +# delete, sync after some time +run_check $SUDO_HELPER $TOP/btrfs subvolume delete -c snap3 +{ sleep 5; run_check $TOP/btrfs filesystem sync $TEST_MNT; } & + +run_check $SUDO_HELPER $TOP/btrfs subvolume sync . $idtodel + +if run_check_stdout $SUDO_HELPER $TOP/btrfs subvolume list -d . | + grep -q "ID $idtodel.*DELETED"; then + _fail "sync did not wait for the subvolume cleanup" +fi + +run_check $TOP/btrfs filesystem sync $TEST_MNT +run_check $SUDO_HELPER $TOP/btrfs subvolume list -d . + +wait +cd .. + +run_check $SUDO_HELPER umount $TEST_MNT diff --git a/tests/misc-tests/010-convert-delete-ext2-subvol/test.sh b/tests/misc-tests/010-convert-delete-ext2-subvol/test.sh new file mode 100755 index 00000000..451e453a --- /dev/null +++ b/tests/misc-tests/010-convert-delete-ext2-subvol/test.sh @@ -0,0 +1,26 @@ +#!/bin/bash +# +# verify that convert rollback finds the ext2_subvolume intact and fails if it +# was partially deleted + +source $TOP/tests/common + +check_prereq btrfs-convert +check_prereq btrfs-debug-tree +check_prereq btrfs + +setup_root_helper +prepare_test_dev + +run_check truncate -s 2G "$TEST_DEV" +run_check mkfs.ext4 -F "$TEST_DEV" +run_check $TOP/btrfs-convert "$TEST_DEV" +run_check $TOP/btrfs-debug-tree "$TEST_DEV" +run_check_mount_test_dev +run_check $SUDO_HELPER $TOP/btrfs subvolume delete -c "$TEST_MNT/ext2_saved" +run_check_umount_test_dev +run_check $TOP/btrfs-debug-tree "$TEST_DEV" +run_check_stdout $TOP/btrfs-convert --rollback "$TEST_DEV" | + grep -q 'is it deleted' || _fail "unexpected rollback" + +exit 0 diff --git a/tests/misc-tests/011-delete-missing-device/test.sh b/tests/misc-tests/011-delete-missing-device/test.sh new file mode 100755 index 00000000..26645f10 --- /dev/null +++ b/tests/misc-tests/011-delete-missing-device/test.sh @@ -0,0 +1,83 @@ +#!/bin/bash +# make sure that 'missing' is accepted for device deletion + +source $TOP/tests/common + +check_prereq btrfs-show-super +check_prereq mkfs.btrfs +check_prereq btrfs + +ndevs=4 +declare -a devs +dev1= +devtodel= + +setup_root_helper + +prepare_devices() +{ + for i in `seq $ndevs`; do + touch img$i + chmod a+rw img$i + truncate -s0 img$i + truncate -s2g img$i + devs[$i]=`run_check_stdout $SUDO_HELPER losetup --find --show img$i` + done +} + +cleanup_devices() +{ + for dev in ${devs[@]}; do + run_mayfail $SUDO_HELPER losetup -d $dev + done + for i in `seq $ndevs`; do + truncate -s0 img$i + done + run_check $SUDO_HELPER losetup --list +} + +test_do_mkfs() +{ + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f $@ ${devs[@]} + run_check $TOP/btrfs-show-super $dev1 + run_check $SUDO_HELPER $TOP/btrfs check $dev1 + run_check $TOP/btrfs filesystem show +} + +test_wipefs() +{ + run_check wipefs -a $devtodel + run_check $SUDO_HELPER losetup -d $devtodel + run_check losetup -a + run_check $TOP/btrfs filesystem show +} +test_delete_missing() +{ + run_check_mount_test_dev -o degraded + run_check $SUDO_HELPER $TOP/btrfs filesystem show $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs device delete missing $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs filesystem show $TEST_MNT + run_check_umount_test_dev + + run_check_mount_test_dev + local out + out="$(run_check_stdout $SUDO_HELPER $TOP/btrfs filesystem show $TEST_MNT)" + if echo "$out" | grep -q -- "$devtodel"; then + _fail "device $devtodel not deleted" + fi + if echo "$out" | grep -q missing; then + _fail "missing device still present" + fi + run_check_umount_test_dev +} + +prepare_devices +dev1=${devs[1]} +devtodel=${devs[3]} +TEST_DEV=$dev1 + +test_do_mkfs +test_wipefs +test_delete_missing + +cleanup_devices diff --git a/tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image b/tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image Binary files differnew file mode 100644 index 00000000..7bf6c509 --- /dev/null +++ b/tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image diff --git a/tests/misc-tests/012-find-root-no-result/test.sh b/tests/misc-tests/012-find-root-no-result/test.sh new file mode 100755 index 00000000..983a8a1e --- /dev/null +++ b/tests/misc-tests/012-find-root-no-result/test.sh @@ -0,0 +1,24 @@ +#!/bin/bash +# Regression test for case btrfs-find-root may print no result on a +# recent fs or balanced fs, whose metadata chunk is the first chunk +# and the only metadata chunk + +source $TOP/tests/common + +check_prereq btrfs-find-root +check_prereq btrfs-image + +run_check $TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \ + _fail "failed to extract first_meta_chunk.btrfs-image" + +result=$(run_check_stdout $TOP/btrfs-find-root test.img | sed '/^Superblock/d') + +if [ -z "$result" ]; then + _fail "btrfs-find-root failed to find tree root" +fi + +if ! echo "$result" | grep -q 'Found tree root at'; then + _fail "btrfs-find-root failed to find tree root, unexpected output" +fi + +rm test.img diff --git a/tests/misc-tests/013-subvolume-sync-crash/test.sh b/tests/misc-tests/013-subvolume-sync-crash/test.sh new file mode 100755 index 00000000..4cb1b4e7 --- /dev/null +++ b/tests/misc-tests/013-subvolume-sync-crash/test.sh @@ -0,0 +1,49 @@ +#!/bin/bash +# +# Verify that subvolume sync waits until the subvolume is cleaned and does not +# crash at the end + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper + +run_check truncate -s 2G $IMAGE +run_check $TOP/mkfs.btrfs -f $IMAGE +run_check $SUDO_HELPER mount $IMAGE $TEST_MNT +run_check $SUDO_HELPER chmod a+rw $TEST_MNT + +cd $TEST_MNT + +for i in `seq 5`; do + run_check dd if=/dev/zero of=file$i bs=1M count=10 +done + +# 128 is minimum +for sn in `seq 130`;do + run_check $SUDO_HELPER $TOP/btrfs subvolume snapshot . snap$sn + for i in `seq 10`; do + run_check dd if=/dev/zero of=snap$sn/file$i bs=1M count=1 + done +done + +run_check $SUDO_HELPER $TOP/btrfs subvolume list . +run_check $SUDO_HELPER $TOP/btrfs subvolume list -d . + +idtodel=`run_check_stdout $SUDO_HELPER $TOP/btrfs inspect-internal rootid snap3` + +# delete, sync after some time +run_check $SUDO_HELPER $TOP/btrfs subvolume delete -c snap* +{ sleep 5; run_check $TOP/btrfs filesystem sync $TEST_MNT; } & + +run_check $SUDO_HELPER $TOP/btrfs subvolume sync . + +run_check $TOP/btrfs filesystem sync $TEST_MNT +run_check $SUDO_HELPER $TOP/btrfs subvolume list -d . + +wait +cd .. + +run_check $SUDO_HELPER umount $TEST_MNT diff --git a/tests/misc-tests/014-filesystem-label/test.sh b/tests/misc-tests/014-filesystem-label/test.sh new file mode 100755 index 00000000..a5e08ccc --- /dev/null +++ b/tests/misc-tests/014-filesystem-label/test.sh @@ -0,0 +1,69 @@ +#!/bin/bash +# +# test label settings + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper + +run_check truncate -s 2G $IMAGE +run_check $TOP/mkfs.btrfs -L BTRFS-TEST-LABEL -f $IMAGE +run_check $SUDO_HELPER mount $IMAGE $TEST_MNT +run_check $SUDO_HELPER chmod a+rw $TEST_MNT + +cd $TEST_MNT +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT +# shortest label +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT a +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT '' + +longlabel=\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +0123456789\ +\ +01234 + +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT "$longlabel" +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT +# 256, must fail +run_mustfail "label 256 bytes long succeeded" \ + $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT "$longlabel"5 +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT +run_mustfail "label 2 * 255 bytes long succeeded" \ + $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT "$longlabel$longlabel" +run_check $SUDO_HELPER $TOP/btrfs filesystem label $TEST_MNT + +cd .. + +run_check $SUDO_HELPER umount $TEST_MNT diff --git a/tests/mkfs-tests.sh b/tests/mkfs-tests.sh new file mode 100755 index 00000000..1afc0282 --- /dev/null +++ b/tests/mkfs-tests.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# +# mkfs.btrfs tests + +LANG=C +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +TEST_DEV=${TEST_DEV:-} +RESULTS="$TOP/tests/mkfs-tests-results.txt" +IMAGE="$TOP/tests/test.img" + +source $TOP/tests/common + +export TOP +export RESULTS +export LANG +export IMAGE +export TEST_DEV + +rm -f $RESULTS + +check_prereq mkfs.btrfs +check_prereq btrfs + +# The tests are driven by their custom script called 'test.sh' + +for i in $(find $TOP/tests/mkfs-tests -maxdepth 1 -mindepth 1 -type d \ + ${TEST:+-name "$TEST"} | sort) +do + echo " [TEST/mkfs] $(basename $i)" + cd $i + echo "=== Entering $i" >> $RESULTS + if [ -x test.sh ]; then + ./test.sh + if [ $? -ne 0 ]; then + _fail "test failed for case $(basename $i)" + fi + fi + cd $TOP +done diff --git a/tests/mkfs-tests/001-basic-profiles/test.sh b/tests/mkfs-tests/001-basic-profiles/test.sh new file mode 100755 index 00000000..a6769214 --- /dev/null +++ b/tests/mkfs-tests/001-basic-profiles/test.sh @@ -0,0 +1,93 @@ +#!/bin/bash +# test various blockgroup profile combinations, use loop devices as block +# devices + +source $TOP/tests/common + +check_prereq btrfs-show-super +check_prereq mkfs.btrfs +check_prereq btrfs + +ndevs=4 +declare -a devs +dev1= + +setup_root_helper + +prepare_devices() +{ + for i in `seq $ndevs`; do + touch img$i + chmod a+rw img$i + truncate -s0 img$i + truncate -s2g img$i + devs[$i]=`run_check_stdout $SUDO_HELPER losetup --find --show img$i` + done +} + +cleanup_devices() +{ + for dev in ${devs[@]}; do + run_check $SUDO_HELPER losetup -d $dev + done + for i in `seq $ndevs`; do + truncate -s0 img$i + done + run_check $SUDO_HELPER losetup --list +} + +test_get_info() +{ + run_check $TOP/btrfs-show-super $dev1 + run_check $SUDO_HELPER $TOP/btrfs check $dev1 + run_check $SUDO_HELPER mount $dev1 $TEST_MNT + run_check $TOP/btrfs filesystem df $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs filesystem usage $TEST_MNT + run_check $SUDO_HELPER $TOP/btrfs device usage $TEST_MNT + run_check $SUDO_HELPER umount "$TEST_MNT" +} +test_do_mkfs() +{ + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f \ + $@ +} + +test_mkfs_single() +{ + test_do_mkfs $@ $dev1 + test_get_info +} +test_mkfs_multi() +{ + test_do_mkfs $@ ${devs[@]} + test_get_info +} + +prepare_devices +dev1=${devs[1]} + +test_mkfs_single +test_mkfs_single -d single -m single +test_mkfs_single -d single -m single --mixed +test_mkfs_single -d single -m dup +test_mkfs_single -d dup -m single +test_mkfs_single -d dup -m dup +test_mkfs_single -d dup -m dup --mixed + +test_mkfs_multi +test_mkfs_multi -d single -m single +test_mkfs_multi -d single -m single --mixed +test_mkfs_multi -d raid0 -m raid0 +test_mkfs_multi -d raid0 -m raid0 --mixed +test_mkfs_multi -d raid1 -m raid1 +test_mkfs_multi -d raid1 -m raid1 --mixed +test_mkfs_multi -d raid10 -m raid10 +test_mkfs_multi -d raid10 -m raid10 --mixed +test_mkfs_multi -d raid5 -m raid5 +test_mkfs_multi -d raid5 -m raid5 --mixed +test_mkfs_multi -d raid6 -m raid6 +test_mkfs_multi -d raid6 -m raid6 --mixed +test_mkfs_multi -d dup -m dup +test_mkfs_multi -d dup -m dup --mixed + +cleanup_devices diff --git a/tests/mkfs-tests/002-no-force-mixed-on-small-volume/test.sh b/tests/mkfs-tests/002-no-force-mixed-on-small-volume/test.sh new file mode 100755 index 00000000..855fbd18 --- /dev/null +++ b/tests/mkfs-tests/002-no-force-mixed-on-small-volume/test.sh @@ -0,0 +1,13 @@ +#!/bin/bash +# +# Verify that we do not force mixed block groups on small volumes anymore + +source $TOP/tests/common + +check_prereq mkfs.btrfs + +setup_root_helper + +run_check truncate -s 512M $IMAGE +mixed=$(run_check_stdout $TOP/mkfs.btrfs -n 64k -f $IMAGE | egrep 'Data|Metadata') +echo "$mixed" | grep -q -v 'Data+Metadata:' || _fail "unexpected: created a mixed-bg filesystem" diff --git a/tests/mkfs-tests/003-mixed-with-wrong-nodesize/test.sh b/tests/mkfs-tests/003-mixed-with-wrong-nodesize/test.sh new file mode 100755 index 00000000..289d5ff0 --- /dev/null +++ b/tests/mkfs-tests/003-mixed-with-wrong-nodesize/test.sh @@ -0,0 +1,12 @@ +#!/bin/bash +# +# Mixed mode needs equal sectorsize and nodesize + +source $TOP/tests/common + +check_prereq mkfs.btrfs + +run_check truncate -s 512M $IMAGE +run_mayfail $TOP/mkfs.btrfs -f -M -s 4096 -n 16384 "$IMAGE" && _fail + +exit 0 diff --git a/tests/mkfs-tests/004-rootdir-keeps-size/test.sh b/tests/mkfs-tests/004-rootdir-keeps-size/test.sh new file mode 100755 index 00000000..7038c8ea --- /dev/null +++ b/tests/mkfs-tests/004-rootdir-keeps-size/test.sh @@ -0,0 +1,31 @@ +#!/bin/bash +# make sure that mkfs.btrfs --rootsize does not change size of the image + +source $TOP/tests/common + +check_prereq mkfs.btrfs + +prepare_test_dev + +test_mkfs_with_size() { + local size + local imgsize + local tmp + + size="$1" + run_check truncate -s$size $TEST_DEV + imgsize=$(run_check_stdout stat --format=%s $TEST_DEV) + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f \ + --rootdir $TOP/Documentation \ + $TEST_DEV + tmp=$(run_check_stdout stat --format=%s $TEST_DEV) + if ! [ "$imgsize" = "$tmp" ]; then + _fail "image size changed from $imgsize to $tmp" + fi +} + +test_mkfs_with_size 128M +test_mkfs_with_size 256M +test_mkfs_with_size 512M +test_mkfs_with_size 1G +test_mkfs_with_size 2G diff --git a/tests/mkfs-tests/005-long-device-name-for-ssd/test.sh b/tests/mkfs-tests/005-long-device-name-for-ssd/test.sh new file mode 100755 index 00000000..cff495e6 --- /dev/null +++ b/tests/mkfs-tests/005-long-device-name-for-ssd/test.sh @@ -0,0 +1,41 @@ +#!/bin/bash +# a long device name must pass the SSD test + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs-show-super + +setup_root_helper +prepare_test_dev + +# prep device +dmname=\ +btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA +dmdev=/dev/mapper/$dmname + +run_check truncate -s0 img +chmod a+w img +run_check truncate -s2g img + +loopdev=`run_check_stdout $SUDO_HELPER losetup --find --show img` +run_check $SUDO_HELPER dmsetup create $dmname --table "0 1048576 linear $loopdev 0" + +dmbase=`readlink -f $dmdev` +base=`basename "$dmbase"` +rot=/sys/class/block/$base/queue/rotational + +# switch rotational +run_check cat $rot +echo 0 | run_check $SUDO_HELPER tee $rot +run_check cat $rot + +# test +run_check_stdout $SUDO_HELPER $TOP/mkfs.btrfs -f $@ $dmdev | + grep -q 'SSD detected:.*yes' || _fail 'SSD not detected' +run_check $TOP/btrfs-show-super $dmdev + +# cleanup +run_check $SUDO_HELPER dmsetup remove $dmname +run_mayfail $SUDO_HELPER losetup -d $loopdev +run_check truncate -s0 img diff --git a/tests/mkfs-tests/006-partitioned-loopdev/partition-1g-1g b/tests/mkfs-tests/006-partitioned-loopdev/partition-1g-1g Binary files differnew file mode 100644 index 00000000..eb057769 --- /dev/null +++ b/tests/mkfs-tests/006-partitioned-loopdev/partition-1g-1g diff --git a/tests/mkfs-tests/006-partitioned-loopdev/test.sh b/tests/mkfs-tests/006-partitioned-loopdev/test.sh new file mode 100755 index 00000000..7c9fb829 --- /dev/null +++ b/tests/mkfs-tests/006-partitioned-loopdev/test.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# recognize partitioned loop devices + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs-show-super + +setup_root_helper + +run_check truncate -s0 img +chmod a+w img +cp partition-1g-1g img +run_check truncate -s2g img + +loopdev=$(run_check_stdout $SUDO_HELPER losetup --partscan --find --show img) +base=$(basename $loopdev) + +# expect partitions named like loop0p1 etc +for looppart in $(ls /dev/$base?*); do + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f $looppart + run_check $TOP/btrfs-show-super $looppart +done + +# cleanup +run_check $SUDO_HELPER losetup -d $loopdev +run_check truncate -s0 img diff --git a/tests/mkfs-tests/007-mix-nodesize-sectorsize/test.sh b/tests/mkfs-tests/007-mix-nodesize-sectorsize/test.sh new file mode 100755 index 00000000..d5374cbd --- /dev/null +++ b/tests/mkfs-tests/007-mix-nodesize-sectorsize/test.sh @@ -0,0 +1,36 @@ +#!/bin/bash +# iterate over nodesize and sectorsize combinations + +source $TOP/tests/common + +check_prereq btrfs-show-super +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper +prepare_test_dev + +test_mkfs_single() +{ + run_check $SUDO_HELPER $TOP/mkfs.btrfs -f "$@" $TEST_DEV + run_check $TOP/btrfs-show-super $TEST_DEV + run_check $SUDO_HELPER $TOP/btrfs check $TEST_DEV +} + +# default +test_mkfs_single + +# nodesize >= sectorsize +for nodesize in 4096 8192 16384 32768 65536; do + for sectorsize in 4096 8192 16384 32768 65536; do + [ $nodesize -lt $sectorsize ] && continue + test_mkfs_single -n $nodesize -s $sectorsize -d single -m single + test_mkfs_single -n $nodesize -s $sectorsize -d single -m dup + done +done + +# nodesize, mixed mode +for nodesize in 4k 8k 16k 32k 64k; do + test_mkfs_single -n $nodesize -s $nodesize -d single -m single --mixed + test_mkfs_single -n $nodesize -s $nodesize -d dup -m dup --mixed +done diff --git a/tests/mkfs-tests/008-secorsize-nodesize-combination/test.sh b/tests/mkfs-tests/008-secorsize-nodesize-combination/test.sh new file mode 100755 index 00000000..151e7b77 --- /dev/null +++ b/tests/mkfs-tests/008-secorsize-nodesize-combination/test.sh @@ -0,0 +1,50 @@ +#!/bin/bash +# test various sectorsize and node size combinations +# including valid and invalid ones +# only do mkfs and fsck check, no mounting as +# sub/multi-pagesize is not supported yet + +source $TOP/tests/common + +check_prereq mkfs.btrfs +check_prereq btrfs + +prepare_test_dev + +# disable mixed bg to avoid sectorsize == nodesize check +features="^mixed-bg" + +# caller need to check whether the combination is valid +do_test() +{ + sectorsize=$1 + nodesize=$2 + run_mayfail $TOP/mkfs.btrfs -O $features -n $nodesize -s $sectorsize \ + $TEST_DEV + ret=$? + if [ $ret == 0 ]; then + run_check $TOP/btrfs check $TEST_DEV + fi + return $ret +} + +# Invalid: Unaligned sectorsize and nodesize +do_test 8191 8191 && _fail + +# Invalid: Aligned sectorsize with unaligned nodesize +do_test 4k 16385 && _fail + +# Invalid: Unaligned sectorsize with aligned nodesize +do_test 8191 16k && _fail + +# Valid: Aligned sectorsize and nodesize +do_test 4k 16k || _fail + +# Invalid: Sectorsize larger than nodesize +do_test 8k 4k && _fail + +# Invalid: too large nodesize +do_test 16k 128k && _fail + +# Valid: large sectorsize +do_test 64k 64k || _fail diff --git a/tests/test-console.sh b/tests/test-console.sh new file mode 100755 index 00000000..cc1cdf3c --- /dev/null +++ b/tests/test-console.sh @@ -0,0 +1,23 @@ +#!/bin/sh +# a shell with test environment set up, logged commands and output + +LANG=C +SCRIPT_DIR=$(dirname $(readlink -f $0)) +TOP=$(readlink -f $SCRIPT_DIR/../) +TEST_DEV=${TEST_DEV:-} +RESULTS="$TOP/tests/test-console.txt" +IMAGE="$TOP/tests/test.img" + +source common +source common.convert + +setup_root_helper + +echo "Eval loop in test environment (log: $RESULTS)" +echo -e " ---------------------\nStarting session, `date`" >> "$RESULTS" +echo -n "`pwd`> " +while read x; do + echo "COMMAND: $x" >> "$RESULTS" + { eval $x; } 2>&1 | tee -a "$RESULTS" + echo -n "`pwd`> " +done |