Sample Header Ad - 728x90

duplicity resends all the data if filename is renamed

2 votes
1 answer
328 views
Let's create a test/ directory containing a random 1 GB file: head -c 1G test/1GBfile, and let's do a backup with [duplicity](http://duplicity.nongnu.org/) : duplicity test/ file:///home/www/backup/ Then /home/www/backup/ contains an encrypted archive, taking around ~1 GB. Then let's add a new file of a few bytes: echo "hello" >test/hello.txt, and redo the backup: duplicity test/ file:///home/www/backup/ The backup/ is still ~ 1 GB. Only a few files were created of < 1 KB, as usual in incremental backup. **Now let's rename the 1 GB file: mv test/1GBfile test/1GBfile_newname and redo the incremental backup**: duplicity test/ file:///home/www/backup/ **Then backup/ is now ~ 2 GB!**. Why does duplicity not take into account the fact it's the same file content with a new name? Here if we had used networking, we would have wasted 1 GB transfer even if the file content is exactly the same. duplicity uses rsync which usually takes care of this problem, is there an option to avoid this problem? ____ Log after the addition of the .txt file:
--------------[ Backup Statistics ]--------------
StartTime 1605543432.43 (Mon Nov 16 17:17:12 2020)
EndTime 1605543432.72 (Mon Nov 16 17:17:12 2020)
ElapsedTime 0.29 (0.29 seconds)
SourceFiles 3
SourceFileSize 1073745926 (1.00 GB)
NewFiles 2
NewFileSize 4102 (4.01 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 2
RawDeltaSize 6 (6 bytes)
TotalDestinationSizeChange 230 (230 bytes)
Errors 0
-------------------------------------------------
Log after the renaming of the file:
--------------[ Backup Statistics ]--------------
StartTime 1605543625.97 (Mon Nov 16 17:20:25 2020)
EndTime 1605543840.72 (Mon Nov 16 17:24:00 2020)
ElapsedTime 214.76 (3 minutes 34.76 seconds)
SourceFiles 3
SourceFileSize 1073745926 (1.00 GB)
NewFiles 2
NewFileSize 1073745920 (1.00 GB)
DeletedFiles 1
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 3
RawDeltaSize 1073741824 (1.00 GB)
TotalDestinationSizeChange 1080871987 (1.01 GB)
Errors 0
-------------------------------------------------
TotalDestinationSizeChange 1080871987 (1.01 GB), arghh! The file has just been *renamed*!
Asked by Basj (2579 rep)
Nov 16, 2020, 04:11 PM
Last activity: Nov 18, 2020, 11:05 AM