Sample Header Ad - 728x90

FIDEDUPERANGE ioctl doesn't behave as expected on btrfs

3 votes
1 answer
443 views
According to [ioctl_fideduperange](https://www.man7.org/linux/man-pages/man2/ioctl_fideduperange.2.html) , > The maximum size of *src_length* is filesystem dependent and is typically 16 MiB. However, I've been able use src_length of > 1 Gib successfully with a single call to ioctl.  Is that warning about 16 MiB just a complete exaggeration, at least for btrfs? Also, according to [the VFS documentation](https://www.kernel.org/doc/html/latest/filesystems/vfs.html) , > Implementations must handle callers passing in len == 0; this means “remap to the end of the source file”. However, when I try setting src_length to 0, the ioctl call succeeds but without doing anything. Am I misreading these two sentences, or does the btrfs implementation simply not conform (well) to the documentation? I'm testing on Linux Mint 20 with kernel 5.4.0-62-generic. I'm using filefrag -sv FILE1 FILE2 to check the block-level allocation of the files, to see if they're duplicated or not. I'm using the program below to deduplicate the files. The files in question are on a RAID-1 btrfs filesystem (on LUKS-encrypted partitions) created with sudo mkfs.btrfs -mraid1 -draid1 /dev/mapper/sda1_crypt /dev/mapper/sdb1_crypt. Scenario:
$ cp -f file1 file2
$ filefrag -sv file1 file2   # see that files use different extents (are not deduplicated)
$ myprog file1 file2
$ filefrag -sv file1 file2   # see that files use the same extents (have been deduplicated)
Program to deduplicate two files:
// deduplicate srcfile and targetfile if contents are identical
// usage:  myprog srcfile targetfile
// compile with:  gcc myprog.c -o myprog

#include 
#include 
#include 
#include 
#include 
#include 
#include 

int main(int argc, char**argv)
{
   struct stat st;
   long size;
   __u64 buf;        /* __u64 for proper field alignment */
   struct file_dedupe_range *range = (struct file_dedupe_range *)buf;

   memset(range, 0, sizeof(struct file_dedupe_range));
   memset(&range->info, 0, sizeof(struct file_dedupe_range_info));

   long srcfd = open(argv, O_RDONLY);
   if (srcfd src_offset = 0;
   range->src_length = size;
// range->src_length = 0;                    // I expected this to work
   range->dest_count = 1;
   range->info.dest_fd = tgtfd;
   range->info.dest_offset = 0;

   while (range->src_length > 0) {
      if (ioctl(srcfd, FIDEDUPERANGE, range) info.bytes_deduped);
      fprintf(stderr, "status: %d\n", range->info.status);
      if (range->info.status == FILE_DEDUPE_RANGE_DIFFERS) {
         fprintf(stderr, "DIFFERS\n");
         break;
      } else if (range->info.status == FILE_DEDUPE_RANGE_SAME) {
         fprintf(stderr, "SAME\n");
      } else {
         fprintf(stderr, "ERROR\n");
         break;
      }

      if (range->info.bytes_deduped >= range->src_length) { break; }
      range->src_length -= range->info.bytes_deduped;
      range->src_offset += range->info.bytes_deduped;
      range->info.dest_offset += range->info.bytes_deduped;
   }
   exit(0);
}
Asked by jrw32982 (1089 rep)
Jan 18, 2021, 10:22 PM
Last activity: Jan 23, 2022, 01:33 AM