FIDEDUPERANGE ioctl doesn't behave as expected on btrfs
3
votes
1
answer
443
views
According to [ioctl_fideduperange](https://www.man7.org/linux/man-pages/man2/ioctl_fideduperange.2.html) ,
> The maximum size of *
src_length
* is filesystem dependent
and is typically 16 MiB.
However, I've been able use src_length
of > 1 Gib successfully with a single call to ioctl
. Is that warning about 16 MiB just a complete exaggeration, at least for btrfs
?
Also, according to [the VFS documentation](https://www.kernel.org/doc/html/latest/filesystems/vfs.html) ,
> Implementations must handle callers passing in len == 0; this means “remap to the end of the source file”.
However, when I try setting src_length
to 0
, the ioctl
call succeeds but without doing anything.
Am I misreading these two sentences, or does the btrfs
implementation simply not conform (well) to the documentation? I'm testing on Linux Mint 20 with kernel 5.4.0-62-generic
. I'm using filefrag -sv FILE1 FILE2
to check the block-level allocation of the files, to see if they're duplicated or not. I'm using the program below to deduplicate the files. The files in question are on a RAID-1 btrfs
filesystem (on LUKS-encrypted partitions) created with sudo mkfs.btrfs -mraid1 -draid1 /dev/mapper/sda1_crypt /dev/mapper/sdb1_crypt
.
Scenario:
$ cp -f file1 file2
$ filefrag -sv file1 file2 # see that files use different extents (are not deduplicated)
$ myprog file1 file2
$ filefrag -sv file1 file2 # see that files use the same extents (have been deduplicated)
Program to deduplicate two files:
// deduplicate srcfile and targetfile if contents are identical
// usage: myprog srcfile targetfile
// compile with: gcc myprog.c -o myprog
#include
#include
#include
#include
#include
#include
#include
int main(int argc, char**argv)
{
struct stat st;
long size;
__u64 buf; /* __u64 for proper field alignment */
struct file_dedupe_range *range = (struct file_dedupe_range *)buf;
memset(range, 0, sizeof(struct file_dedupe_range));
memset(&range->info, 0, sizeof(struct file_dedupe_range_info));
long srcfd = open(argv, O_RDONLY);
if (srcfd src_offset = 0;
range->src_length = size;
// range->src_length = 0; // I expected this to work
range->dest_count = 1;
range->info.dest_fd = tgtfd;
range->info.dest_offset = 0;
while (range->src_length > 0) {
if (ioctl(srcfd, FIDEDUPERANGE, range) info.bytes_deduped);
fprintf(stderr, "status: %d\n", range->info.status);
if (range->info.status == FILE_DEDUPE_RANGE_DIFFERS) {
fprintf(stderr, "DIFFERS\n");
break;
} else if (range->info.status == FILE_DEDUPE_RANGE_SAME) {
fprintf(stderr, "SAME\n");
} else {
fprintf(stderr, "ERROR\n");
break;
}
if (range->info.bytes_deduped >= range->src_length) { break; }
range->src_length -= range->info.bytes_deduped;
range->src_offset += range->info.bytes_deduped;
range->info.dest_offset += range->info.bytes_deduped;
}
exit(0);
}
Asked by jrw32982
(1089 rep)
Jan 18, 2021, 10:22 PM
Last activity: Jan 23, 2022, 01:33 AM
Last activity: Jan 23, 2022, 01:33 AM