summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* btrfs-progs: print-tree: Do correct offset output for ROOT_ITEMQu Wenruo2018-03-30
| | | | | | | | | | | | | | | | | Commit Fixes: 8c36786c8198 ("btrfs-progs: print-tree: Print offset as tree objectid for ROOT_ITEM") changes how we translate offset of ROOT_ITEM. However the fact is, even for ROOT_ITEM, we have different meaning of offset. For tree reloc tree, it's indeed subvolume id. But for other trees, it's the transid of when it's created. Reported-by: Nikolay Borisov <nborisov@suse.com> Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Nikolay Borisov <nborisov@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: convert/ext2: Remove check for ext2_ext_attr_entry->e_value_blockQu Wenruo2018-03-30
| | | | | | | | | | | | | In latest e2fsprogs (1.44.0) definition of ext2_ext_attr_entry has removed member e_value_block, as currently ext* doesn't support it set anyway. So remove such check so that we can pass compile. Issue: #110 Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=199071 Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* Btrfs-progs: add fsck test for filesystem with shared prealloc extentsFilipe Manana2018-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Verify that a filesystem check operation (fsck) does not report the following scenario as an error: An extent is shared between two inodes, as a result of clone/reflink operation, and for one of the inodes, lets call it inode A, the extent is referenced through a file extent item as a prealloc extent, while for the other inode, call it inode B, the extent is referenced through a regular file extent item, that is, it was written to. The goal of this test is to make sure a filesystem check operation will not report "odd csum items" errors for the prealloc extent at inode A, because this scenario is valid since the extent was written through inode B and therefore it is expected to have checksum items in the filesystem's checksum btree for that shared extent. Such scenario can be created with the following steps for example: mkfs.btrfs -f /dev/sdb mount /dev/sdb /mnt touch /mnt/foo xfs_io -c "falloc 0 256K" /mnt/foo sync xfs_io -c "pwrite -S 0xab 0 256K" /mnt/foo touch /mnt/bar xfs_io -c "reflink /mnt/foo 0 0 256K" /mnt/bar xfs_io -c "fsync" /mnt/bar <power fail> mount /dev/sdb /mnt umount /mnt This scenario is fixed by the following patch for the filesystem checker: "Btrfs-progs: check, fix false error reports for shared prealloc extents" Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* Btrfs-progs: check, fix false error reports for shared prealloc extentsFilipe Manana2018-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Under some cases the filesystem checker reports an error when it finds checksum items for an extent that is referenced by an inode as a prealloc extent. Such cases are not an error when the extent is actually shared (was cloned/reflinked) with other inodes and was written through one of those other inodes. Example: $ mkfs.btrfs -f /dev/sdb $ mount /dev/sdb /mnt $ touch /mnt/foo $ xfs_io -c "falloc 0 256K" /mnt/foo $ sync $ xfs_io -c "pwrite -S 0xab 0 256K" /mnt/foo $ touch /mnt/bar $ xfs_io -c "reflink /mnt/foo 0 0 256K" /mnt/bar $ xfs_io -c "fsync" /mnt/bar <power fail> $ mount /dev/sdb /mnt $ umount /mnt $ btrfs check /dev/sdc Checking filesystem on /dev/sdb UUID: 52d3006e-ee3b-40eb-aa21-e56253a03d39 checking extents checking free space cache checking fs roots root 5 inode 257 errors 800, odd csum item ERROR: errors found in fs roots found 688128 bytes used, error(s) found total csum bytes: 256 total tree bytes: 163840 total fs tree bytes: 65536 total extent tree bytes: 16384 btree space waste bytes: 138819 file data blocks allocated: 10747904 referenced 10747904 $ echo $? 1 So teach check to not report such cases as errors by checking if the extent is shared with other inodes and if so, consider it an error the existence of checksum items only if all those other inodes are referencing the extent as a prealloc extent. This case can be hit often when running the generic/475 testcase from fstests. A test case will follow in a separate patch. Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Nikolay Borisov <nborisov@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: ctree: Add extra level check for read_node_slot()Qu Wenruo2018-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Strangely, we have level check in btrfs_print_tree() while we don't have the same check in read_node_slot(). That's to say, for the following corruption, btrfs_search_slot() or btrfs_next_leaf() can return invalid leaf: Parent eb: node XXXXXX level 1 ^^^^^^^ Child should be leaf (level 0) ... key (XXX XXX XXX) block YYYYYY Child eb: leaf YYYYYY level 1 ^^^^^^^ Something went wrong now And for the corrupted leaf returned, later caller can be screwed up easily. Reported-by: Ralph Gauges <ralphgauges@googlemail.com> Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Nikolay Borisov <nborisov@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: tests: fix source path for testsuiteDavid Sterba2018-03-30
| | | | | | | | The commit cebf3b37228cbde730a5 ("btrfs-progs: introduce TEST_TOP and INTERNAL_BIN for tests") did not convert all test paths. This would break the exported testsutie. Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: tests: Add test case with valid orphan inodeQu Wenruo2018-03-30
| | | | | | | | Regression test for false alerts in lowmem mode. Signed-off-by: Qu Wenruo <wqu@suse.com> [ update test ] Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: check: lowmem: Fix false alert about orphan inodeQu Wenruo2018-03-30
| | | | | | | | | | | | | | | | | | | | Btrfs can delay inode deletion and in that case btrfs will unlink the victim inode from its parent dir, and insert a marker to info btrfs to delete it later. In that case, such victim inode will have nlinks == 0, but is still completely valid. Original mode won't report such problem, but lowmem mode doesn't check the ORPHAN_ITEM key for such inode, and can report false alert like: ------ ERROR: root 257 INODE[28891726] is orphan item ------ Fix such false alert by checking orphan item for inode whose nlink is 0. Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: gitignore: Ignore patchesQu Wenruo2018-03-30
| | | | | | | | | | | Although it's not a good practice to format all patches under project directory, sometimes lazy bones like me just like to put patches under project directory. Just ignore such patches. Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: gitignore: Ignore *.restored test imageQu Wenruo2018-03-30
| | | | | Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: fsck-tests: Introduce test case with keyed data backref with ↵Lu Fengqi2018-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the extent offset Add the testcase for false alert of data extent backref lost with the extent offset. The image can be reproduced by the following commands: ------ dev=~/test.img mnt=/mnt/btrfs umount $mnt &> /dev/null fallocate -l 128M $dev mkfs.btrfs $dev mount $dev $mnt for i in `seq 1 10`; do xfs_io -f -c "pwrite 0 2K" $mnt/file$i done xfs_io -f -c "falloc 0 64K" $mnt/file11 for i in `seq 1 32`; do xfs_io -f -c "reflink $mnt/file11 0 $(($i * 64))K 64K" $mnt/file11 done xfs_io -f -c "reflink $mnt/file11 32K $((33 * 64))K 32K" $mnt/file11 btrfs subvolume snapshot $mnt $mnt/snap1 umount $mnt btrfs-image -c9 $dev extent_data_ref.img ------ Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: check/lowmem: Fix false alert of data extent backref lost for ↵Lu Fengqi2018-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | snapshot Btrfs lowmem check reports the following false alert: ------ ERROR: file extent[267 2162688] root 256 owner 5 backref lost ------ The file extent is in the leaf which is shared by file tree 256 and fs tree. ------ leaf 30605312 items 46 free space 4353 generation 7 owner 5 ...... item 45 key (267 EXTENT_DATA 2162688) itemoff 5503 itemsize 53 generation 7 type 2 (prealloc) prealloc data disk byte 13631488 nr 65536 prealloc data offset 32768 nr 32768 ------ And there is the corresponding extent_data_ref item in the extent tree. ------ item 1 key (13631488 EXTENT_DATA_REF 1007496934287921081) itemoff 15274 itemsize 28 extent data backref root 5 objectid 267 offset 2129920 count 1 ------ The offset of EXTENT_DATA_REF which is the hash of the owner root objectid, the inode number and the calculated offset (file offset - extent offset). What caused the false alert is the code mix up the owner root objectid and the file tree objectid. Fixes: b0d360b541f0 ("btrfs-progs: check: introduce function to check data backref in extent tree") Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: check/lowmem: Fix the incorrect error message of ↵Lu Fengqi2018-03-30
| | | | | | | | | | | | | check_extent_data_item Instead of the disk_bytenr and disk_num_bytes of the extent_item which the file extent references, we should output the objectid and offset of the file extent. And the leaf may be shared by the file trees, we should print the objectid of the root and the owner of the leaf. Fixes: b0d360b541f0 ("btrfs-progs: check: introduce function to check data backref in extent tree") Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: dump-super: Don't verify csum if csum type or size is unknownQu Wenruo2018-03-30
| | | | | | Reported-by: Ken Swenson <flat@imo.uto.moe> Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: free-space-cache: Enhance free space cache free space checkQu Wenruo2018-03-30
| | | | | | | | | | | | | | | | | | | | | | | When we found free space difference between free space cache and block group item, we just discard this free space cache. Normally such difference is caused by btrfs_reserve_extent() called by delalloc which is out of a transaction. And since all btrfs_release_extent() is called with a transaction, under heavy race free space cache can have less free space than block group item. Normally kernel will detect such difference and just discard that cache. However we must be more careful if free space cache has more free space cache, and if that happens, paried with above race one invalid free space cache can be loaded into kernel. So if we find any free space cache who has more free space then block group item, we report it as an error other than ignoring it. Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: docs: add section about filesystem limits to btrfs(5)David Sterba2018-03-30
| | | | Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: free-space-cache: Use DIV_ROUND_UP() to replace open codeQu Wenruo2018-03-30
| | | | | Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: use pkg-config detection for the right Python versionOmar Sandoval2018-03-30
| | | | | | | | The user may have specified a different version of Python than the python3 from their $PATH (e.g., with PYTHON=/usr/bin/python3.6). Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: ci: enable clang and python for musl build testDavid Sterba2018-03-06
| | | | | | | The updated image now provides clang, so the variable is exported from the base environment to docker. And we have python too. Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: ci: update test image packages - add clang and pythonDavid Sterba2018-03-06
| | | | Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutils: add python-devel detectionDavid Sterba2018-03-06
| | | | | | Use pkg-config to detect python devel. Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: ci: cache built dependenciesDavid Sterba2018-03-06
| | | | | | | For a slight speed up of the build, cache reiserfs and zstd. A quick cache validity is done, or it can be cleared manually on travis web UI. Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: replace test_issubvolume() with btrfs_util_is_subvolume()Omar Sandoval2018-03-06
| | | | | | | | | This gets the remaining occurrences that weren't covered by previous conversions. Signed-off-by: Omar Sandoval <osandov@fb.com> [ fixup test_issubvolume due to removed dependency patch ] Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: use libbtrfsutil for subvol syncOmar Sandoval2018-03-06
| | | | | | | | btrfs_util_f_deleted_subvolumes() replaces enumerate_dead_subvols() and btrfs_util_f_subvolume_info() replaces is_subvolume_cleaned(). Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: use libbtrfsutil for subvol showOmar Sandoval2018-03-06
| | | | | | | | Now implemented with btrfs_util_subvolume_path(), btrfs_util_subvolume_info(), and subvolume iterators. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: use libbtrfsutil for subvol deleteOmar Sandoval2018-03-06
| | | | | | | | Most of the interesting part of this command is the commit mode, so this only saves a little bit of code. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: use libbtrfsutil for get-defaultOmar Sandoval2018-03-06
| | | | | | | | | The only thing of note here is the "top level" column. This used to mean something else, but for a long time it has been equal to the parent ID. I preserved this for backwards compatability. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: use libbtrfsutil for set-defaultOmar Sandoval2018-03-06
| | | | | Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: use libbtrfsutil for sync ioctlsOmar Sandoval2018-03-06
| | | | | Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: use libbtrfsutil for read-only propertyOmar Sandoval2018-03-06
| | | | | Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_deleted_subvolumes()Omar Sandoval2018-03-06
| | | | | Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_delete_subvolume()Omar Sandoval2018-03-06
| | | | | | | We also support recursive deletion using a subvolume iterator. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_create_snapshot()Omar Sandoval2018-03-06
| | | | | | | | Thanks to subvolume iterators, we can also implement recursive snapshot fairly easily. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add subvolume iterator helpersOmar Sandoval2018-03-06
| | | | | | | | | | | | This is how we can implement stuff like `btrfs subvol list`. Rather than producing the entire list upfront, the iterator approach uses less memory in the common case where the whole list is not stored (O(max subvolume path length)). It supports both pre-order traversal (useful for, e.g, recursive snapshot) and post-order traversal (useful for recursive delete). Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add stub for reallocarrayDavid Sterba2018-03-06
| | | | | | | This function is new in glibc 2.26 and breaks build in CI and possibly other environments. Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_[gs]et_default_subvolume()Omar Sandoval2018-03-06
| | | | | | | | set_default_subvolume() is a trivial ioctl(), but there's no ioctl() for get_default_subvolume(), so we need to search the root tree. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_[gs]et_read_only()Omar Sandoval2018-03-06
| | | | | | | | | In the future, btrfs_util_[gs]et_subvolume_flags() might be useful, but since these are the only subvolume flags we've defined in all this time, this will do for now. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_subvolume_info()Omar Sandoval2018-03-06
| | | | | | | | This gets the the information in `btrfs subvolume show` from the root item. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_subvolume_path()Omar Sandoval2018-03-06
| | | | | | | | We can just walk up root backrefs with BTRFS_IOC_TREE_SEARCH and inode paths with BTRFS_IOC_INO_LOOKUP. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_create_subvolume()Omar Sandoval2018-03-06
| | | | | | | | Doing the ioctl() directly isn't too bad, but passing in a full path is more convenient than opening the parent and passing the path component. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add btrfs_util_is_subvolume() and btrfs_util_subvolume_id()Omar Sandoval2018-03-06
| | | | | | | | These are the most trivial helpers in the library and will be used to implement several of the more involved functions. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: copy in Btrfs UAPI headersOmar Sandoval2018-03-06
| | | | | | | | | Systems with older kernels won't have these available, and the copies in btrfs-progs aren't quite compatible, so for now, let's just copy these in. We can potentially deduplicate some of this in the future. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: fix Python testsOmar Sandoval2018-03-06
| | | | | | | These were broken when the patch series got shuffled around. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: Move chunk stripe size calculation function to volumes.hQu Wenruo2018-03-06
| | | | | Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: volumes: Allow find_free_dev_extent() to return maximum hole sizeQu Wenruo2018-02-24
| | | | | | | | Just as kernel find_free_dev_extent(), allow it to return maximum hole size for us to build device list for later chunk allocator rework. Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: Introduce btrfs_raid_array and related infrastructuresQu Wenruo2018-02-24
| | | | | | | | | | | As part of the effort to unify code and behavior between btrfs-progs and kernel, copy the btrfs_raid_array from kernel to btrfs-progs. So later we can use the btrfs_raid_array[] to get needed raid info other than manually do if-else branches. Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* btrfs-progs: Refactor parameter of BTRFS_MAX_DEVS() from root to fs_infoQu Wenruo2018-02-24
| | | | | Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add filesystem sync helpersOmar Sandoval2018-02-24
| | | | | | | Namely, sync, start_sync, and wait_sync. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add qgroup inheritance helpersOmar Sandoval2018-02-24
| | | | | | | | | | We want to hide struct btrfs_qgroup_inherit from the user because that comes from the Btrfs UAPI headers. Instead, wrap it in a struct btrfs_util_qgroup_inherit and provide helpers to manipulate it. This will be used for subvolume and snapshot creation. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
* libbtrfsutil: add Python bindingsOmar Sandoval2018-02-24
| | | | | | | | | | | The C libbtrfsutil library isn't very useful for scripting, so we also want bindings for Python. Writing unit tests in Python is also much easier than doing so in C. Only Python 3 is supported; if someone really wants Python 2 support, they can write their own bindings. This commit is just the scaffolding. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: David Sterba <dsterba@suse.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-17 04:09:58 +0000