Sample Header Ad - 728x90

Unix & Linux Stack Exchange

Q&A for users of Linux, FreeBSD and other Unix-like operating systems

Latest Questions

-1 votes
1 answers
32 views
patch fails on unified diff, but works fine with context diff
I am struggling with understanding how `diff` and `patch` are supposed to work. Here's my use case: I use Betterfox user.js config file for my Firefox profile, but I add some overrides to it. I want to save my overrides to a patch file and apply it every time I download a new user.js. Sounds simple...
I am struggling with understanding how diff and patch are supposed to work. Here's my use case: I use Betterfox user.js config file for my Firefox profile, but I add some overrides to it. I want to save my overrides to a patch file and apply it every time I download a new user.js. Sounds simple enough, so I do the following: 1. Download user.js from GitHub 2. Create a copy of it under user.js.orig 3. Edit user.js to add my overrides 4. Run diff -U6 user.js.orig user.js > user.js.patch - I've chosen unified diff because it's the format I'm more familiar with from Git - I've chosen the context size of 6 because it includes a bit more context that is not supposed to change (file section headers) 5. To apply the patch, I run `patch user.js user.js.diff # some time later... rm user.js && wget https://github.com/yokoffing/Betterfox/raw/refs/tags/140.0/user.js PATCH_VERBOSE=1 patch user.js user.js.diff` works just fine. Aren't those supposed to be two ways of conveying the same diff? Why does a context diff work, but a unified one doesn't? **EDIT:** I forgot to specify what software I'm using. I am on macOS 13.7.6 Ventura, and I am using stock /usr/bin/diff and /usr/bin/patch, both claim to be “compliant with the IEEE Std 1003.1-2008 (‘POSIX.1’) specification”
Nikita Karamov (99 rep)
Jul 26, 2025, 12:58 PM • Last activity: Jul 26, 2025, 08:16 PM
91 votes
16 answers
117433 views
How to color diff output?
I wanted to format the Unix files conditionally, I am currently working on `diff` command and wanted to know if it is possible to format the text of the `diff` command output. Example: >Matched values should be displayed in green. Unmatched values should be displayed in red. Suppose I have two files...
I wanted to format the Unix files conditionally, I am currently working on diff command and wanted to know if it is possible to format the text of the diff command output. Example: >Matched values should be displayed in green. Unmatched values should be displayed in red. Suppose I have two files file1 and file2 and my command is diff file1 file2 . Now I wanted that suppose output contain 5 mismatch then those mismatch should be displayed in Red color. How to achieve this using unix? In short "Change color to red for the output of diff command for values which mismatch"
Aman (1281 rep)
Apr 16, 2015, 10:11 AM • Last activity: Jul 25, 2025, 03:26 PM
6 votes
3 answers
12915 views
Show only changed lines without syntax with git diff
I have a list of usernames that are in a text file. I update this file and commit it. I am looking for a way to get a list of the changes since the last commit. I do not want any diff formatting at all, I just want to get an output that has usernames one per line (as they are added to each commit) s...
I have a list of usernames that are in a text file. I update this file and commit it. I am looking for a way to get a list of the changes since the last commit. I do not want any diff formatting at all, I just want to get an output that has usernames one per line (as they are added to each commit) since the last commit. I can't find a setting that will remove all the git diff syntax from the output so it's purely a list new lines added only. # Example Original file: user1 user2 user3 I then add user4 user5 Then commit. I want to be able to do a git diff and see only: user4 user5
Michael (161 rep)
Jun 15, 2018, 09:47 AM • Last activity: Jun 30, 2025, 01:12 PM
6 votes
2 answers
18980 views
How to compare two tar archives (including file content, new/removed files, symlinks)?
I have two tar archives (compressed or not compressed), and I want to find all differences in the two archives. Both archives contain a complete file system (i.e. when unpacked, would generate directories like `/bin`, `/home`, `/root`, `/usr`, `/var`, `/etc`,... I hope you get the point). I want to...
I have two tar archives (compressed or not compressed), and I want to find all differences in the two archives. Both archives contain a complete file system (i.e. when unpacked, would generate directories like /bin, /home, /root, /usr, /var, /etc,... I hope you get the point). I want to have a list of the following: - New files - Removed files - Changed files (content of file, not just size) - Changed symlinks (both relative and absolute) - New/removed symlinks I cannot just unpack those archives and use diff, as diff will not correctly recognize absolute symlinks (as they would point out of the file system structure of the archive). Is there another way to compare the content of two tar archives?
Alex (5848 rep)
Nov 12, 2013, 01:19 PM • Last activity: Jun 5, 2025, 08:57 AM
2 votes
1 answers
155 views
Unix or Linux command to compare binary files
I'm looking for a command that compares binary files. Of course, I know about `diff`, but it is not very good at binaries. I have two files from a error-prone source (scratched dvd) which should be equal but aren't. (Well, realy more than two, and I get about 6 different md5sum out of 15 samples.) I...
I'm looking for a command that compares binary files. Of course, I know about diff, but it is not very good at binaries. I have two files from a error-prone source (scratched dvd) which should be equal but aren't. (Well, realy more than two, and I get about 6 different md5sum out of 15 samples.) I'm looking for a tool that lists the positions where the files differ. --- In addition to the accepted answer, xxd looks good as it can also be used to change back edited hex-files to binary.
Gyro Gearloose (455 rep)
May 9, 2025, 07:25 PM • Last activity: May 14, 2025, 05:02 PM
0 votes
0 answers
162 views
Process substitution doesn't work with diff and node
I'm trying to compare the output of my `fizzbuzz.js` program: for (let i = 1; i <= 100; i++) { let out = ""; if (i % 3 == 0) out += "Fizz"; if (i % 5 == 0) out += "Buzz"; if (i % 3 && i % 5) out = i; console.log(out); } with the contents of a `expected-output.txt` file which contains: 1 2 Fizz 4 Buz...
I'm trying to compare the output of my fizzbuzz.js program: for (let i = 1; i <= 100; i++) { let out = ""; if (i % 3 == 0) out += "Fizz"; if (i % 5 == 0) out += "Buzz"; if (i % 3 && i % 5) out = i; console.log(out); } with the contents of a expected-output.txt file which contains: 1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 [...] I can't get process substitution to work: diff -u expected-output.txt <(node fizzbuzz.js) seems to hang indefinitely. What could be causing this? My environment is macOS Mojave, bash 3.2.57, node v12.5.0. EDIT: this seems to be a macOS issue. Everything works fine on a Ubuntu 18.04 machine with newer bash 4.4.19
Chef Tony (465 rep)
Jul 14, 2019, 06:09 PM • Last activity: Apr 12, 2025, 12:59 PM
4 votes
2 answers
773 views
Find most closely matching files in two directories
I'm trying to find a solution for the following problem. I have two sets of files: - Folder A has around 400 text files. - Folder B has around 20 000 text files total in several subfolders. Files in folder A are either modified versions of files in folder B or they are parts of files in folder B. Wh...
I'm trying to find a solution for the following problem. I have two sets of files: - Folder A has around 400 text files. - Folder B has around 20 000 text files total in several subfolders. Files in folder A are either modified versions of files in folder B or they are parts of files in folder B. When I say "parts", I mean that a file in folder A might contain part of the text of a file in folder B, but not everything. I want to match those pairs i.e. for each file in folder A I want to find the file or files in folder B that most closely resemble the file in folder A. For example I would like to have the following kind of report:
File ./A/foo123.txt most closely matches file ./B/bar243.txt with 68% of lines identical.
File ./A/bar306.txt most closely matches file ./B/foo85.txt with 30% of lines identical.
Is there a command line tool that I could use to achieve this result? Or what would be the best way to do this?
ttsc (141 rep)
Oct 8, 2019, 02:24 PM • Last activity: Apr 5, 2025, 07:30 AM
10 votes
2 answers
20760 views
Diff: only compare if file exists, not contents
I want to use Diff only to check if files and directories exist the same in two locations but NOT compare the contents of the files themselves, because that's all I need and a regular Diff just takes too long for the amount of data. How would I go about this? Is there some other Debian standard tool...
I want to use Diff only to check if files and directories exist the same in two locations but NOT compare the contents of the files themselves, because that's all I need and a regular Diff just takes too long for the amount of data. How would I go about this? Is there some other Debian standard tool that can accomplish this?
morph (121 rep)
Apr 3, 2016, 04:16 PM • Last activity: Apr 3, 2025, 11:26 PM
0 votes
0 answers
40 views
ubuntu system freeze while diff reading, SysRq REISUB not working
I've experienced total system freeze while checking if files where copied correctly by ``` diff -rq [INTERNAL HDD] [USB HDD] ``` This happened twice, always while running the `diff`. On the second occasion I had system monitor up and RAM was about 30% filled and CPU was also empty. I've tried Alt+Sy...
I've experienced total system freeze while checking if files where copied correctly by
diff -rq [INTERNAL HDD] [USB HDD]
This happened twice, always while running the diff. On the second occasion I had system monitor up and RAM was about 30% filled and CPU was also empty. I've tried Alt+SysRq+REISUB to reboot safely but the system was totally unresponsive. Suspecting RAM, I've conducted memtest without any issue. Do someone have any idea while this happens? (I'm a bit reluctant to test it further much as the data are valuable). The size of the data is ~4T in ~60k files. Some system info
INTERNAL
description: EXT4 volume
capabilities: journaled extended_attributes large_files huge_files dir_nlink recover 64bit extents ext4 ext2 initialized
configuration: ansiversion=5 created=2024-02-28 16:06:38 filesystem=ext4 label=18T lastmountpoint=/media/marek/18T logicalsectorsize=512 modified=2025-03-20 08:38:08 mount.fstype=ext4 mount.options=rw,nosuid,nodev,relatime,errors=remount-ro mounted=2025-03-20 08:38:08 sectorsize=4096 state=mounted

# for some reason, lshw reports this hdd to be "size: 380GiB"
# while lsblk reports (expected)
# sda                        8:0    0  16,4T  0 disk   /media/marek/18T


USB
filesystem=ntfs label=Elements mount.fstype=ntfs3 mount.options=rw,nosuid,nodev,relatime,uid=1000,gid=1000,iocharset=utf8 name=Elements state=mounted


SYSTEM
Ubuntu 24.04.2 LTS
Intel® Core™ i7-5820K × 12
RAM 32,0 GiB
Linux 6.8.0-55-generic
Marek Schwarz (101 rep)
Mar 20, 2025, 08:18 AM
13 votes
2 answers
5587 views
Identify duplicate blocks of text within a file
Is there a convenient way to identify duplicate or near duplicate blocks of text within a file? I want to use this for identifying code duplication. It looks like there are specialty programs with this capability but I'm not looking to get that involved. I'm hoping there's a tool similar to diff tha...
Is there a convenient way to identify duplicate or near duplicate blocks of text within a file? I want to use this for identifying code duplication. It looks like there are specialty programs with this capability but I'm not looking to get that involved. I'm hoping there's a tool similar to diff that will can do a sort of "within a file" diff. Even better would be a within a single file vimdiff.
Praxeolitic (1688 rep)
Oct 1, 2014, 02:48 AM • Last activity: Mar 3, 2025, 06:23 PM
2 votes
1 answers
5086 views
How to use tkdiff to compare two gzip files in gui mode?
How to use tkdiff to compare two gzip files in gui mode?
How to use tkdiff to compare two gzip files in gui mode?
Vicky Wu (21 rep)
Jan 11, 2019, 11:13 PM • Last activity: Feb 13, 2025, 01:04 AM
3 votes
4 answers
13885 views
Diff between two csv files based on the column
I have two files file1.csv and file2.csv. Below are the contents of file1.csv AL.jar;d8c06ebedd7954681f34ab5c94fdc4fb AR.jar;9a553dd203d0979aa60004e19cc98c12 BI.jar;8022f6c5f83ba040394ff0b0a0323e8e BV.jar;f53c4a8c988aa8806b54063ebc682803 CaseUtilities.jar;e5f653d899298f5e5d56f357b6f781c5 CO.jar;b2f7...
I have two files file1.csv and file2.csv. Below are the contents of file1.csv AL.jar;d8c06ebedd7954681f34ab5c94fdc4fb AR.jar;9a553dd203d0979aa60004e19cc98c12 BI.jar;8022f6c5f83ba040394ff0b0a0323e8e BV.jar;f53c4a8c988aa8806b54063ebc682803 CaseUtilities.jar;e5f653d899298f5e5d56f357b6f781c5 CO.jar;b2f7a0ab6e646d6793631e5c97e05096 file2.csv AL.jar;d8c06ebedd7954681f34ab5c94fdc4fb AR.jar;4e6e584dd852684ba21ae63990e2a1a6 BV.jar;213d9df82095764702ef4929424a1a0c CaseUtilities.jar;5b787f1f3d57922bd980ebbfe9a5343e CO.jar;cfb994078ff4373c7e0f15de19830a3d Common.jar;a09b520288870aa3888194ce59179dbd We need to compare two files based on the contents. I want to make diff which is based only on values of first column, so the result should be AL.jar;d8c06ebedd7954681f34ab5c94fdc4fb AL.jar;d8c06ebedd7954681f34ab5c94fdc4fb AR.jar;9a553dd203d0979aa60004e19cc98c12 AR.jar;4e6e584dd852684ba21ae63990e2a1a6 BI.jar;8022f6c5f83ba040394ff0b0a0323e8e BV.jar;f53c4a8c988aa8806b54063ebc682803 BV.jar;213d9df82095764702ef4929424a1a0c CaseUtilities.jar;e5f653d899298f5e5d56f357b6f781c5 CaseUtilities.jar;5b787f1f3d57922bd980ebbfe9a5343e CO.jar;b2f7a0ab6e646d6793631e5c97e05096 CO.jar;cfb994078ff4373c7e0f15de19830a3d Common.jar;a09b520288870aa3888194ce59179dbd I have tried the below command diff -y file1.csv file2.csv But the below output is not as expected. AL.jar;d8c06ebedd7954681f34ab5c94fdc4fb AL.jar;d8c06ebedd7954681f34ab5c94fdc4fb AR.jar;9a553dd203d0979aa60004e19cc98c12 | AR.jar;4e6e584dd852684ba21ae63990e2a1a6 BI.jar;8022f6c5f83ba040394ff0b0a0323e8e | BV.jar;213d9df82095764702ef4929424a1a0c BV.jar;f53c4a8c988aa8806b54063ebc682803 | CaseUtilities.jar;5b787f1f3d57922bd980ebbfe9a5343e CaseUtilities.jar;e5f653d899298f5e5d56f357b6f781c5 | CO.jar;cfb994078ff4373c7e0f15de19830a3d CO.jar;b2f7a0ab6e646d6793631e5c97e05096 | Common.jar;a09b520288870aa3888194ce59179dbd Any idea how can my expected output be achieved!
sabarish jackson (628 rep)
Dec 12, 2018, 01:11 PM • Last activity: Feb 12, 2025, 07:10 PM
68 votes
6 answers
95569 views
How to diff files ignoring comments (lines starting with #)?
I've two configuration files, the original from the package manager and a customized one modified by myself. I've added some comments to describe behavior. How can I run `diff` on the configuration files, skipping the comments? A commented line is defined by: - optional leading whitespace (tabs and...
I've two configuration files, the original from the package manager and a customized one modified by myself. I've added some comments to describe behavior. How can I run diff on the configuration files, skipping the comments? A commented line is defined by: - optional leading whitespace (tabs and spaces) - hash sign (#) - anything other character The (simplest) regular expression skipping the first requirement would be #.*. I tried the --ignore-matching-lines=RE (-I RE) option of GNU diff 3.0, but I couldn't get it working with that RE. I also tried .*#.* and .*\#.* without luck. Literally putting the line (Port 631) as RE does not match anything, neither does it help to put the RE between slashes. As suggested in “diff” tool's flavor of regex seems lacking? , I tried grep -G: grep -G '#.*' file This seems to match the comments, but it does not work for diff -I '#.*' file1 file2. So, how should this option be used? How can I make diff skip certain lines (in my case, comments)? Please do not suggest greping the file and comparing the temporary files.
Lekensteyn (21600 rep)
Jul 20, 2011, 01:05 PM • Last activity: Jan 21, 2025, 01:44 PM
1 votes
0 answers
58 views
p4merge open each window minimized when running a git diff/ merge
I've asked the question on SO before, but maybe it is better placed here. I'm running on Ubuntu `24.04.1 LTS` (X11) and have `p4merge` installed in version `p4v-2024.3.2656785`. I'm using `p4merge` since years and I really like it, but recently it started to open **all** windows minimized and if you...
I've asked the question on SO before, but maybe it is better placed here. I'm running on Ubuntu 24.04.1 LTS (X11) and have p4merge installed in version p4v-2024.3.2656785. I'm using p4merge since years and I really like it, but recently it started to open **all** windows minimized and if you want to check the differences you have to maximize the window manually. If you are doing a diff/merge with multiple files, this is **quite annoying**. Maybe this is related to the recent ubuntu upgrade 24.04 -> 24.04.1, but I think the issue started shortly before. In the past it opened the **first** window minimized and after i maximized this window, it then remembered its state and open the following windows in maximized view. In my .gitconfig the diff/merge command for p4merge looks like this: [mergetool "p4m"] cmd = p4merge $BASE $LOCAL $REMOTE $MERGED keepTemporaries = false trustExitCode = false keepBackup = false [difftool "p4m"] cmd = p4merge $LOCAL $REMOTE Any ideas how to fix this?
morecore (111 rep)
Nov 14, 2024, 02:38 PM • Last activity: Jan 16, 2025, 03:40 PM
30 votes
2 answers
45696 views
diff - output line-numbers
I want to use cli tool for file comparison and need line-number before output line with which help I could jump to line difference, because I use tool which understands where to jump, if the line begins like this `:line-number: regular line contents` So I tried `diff`, and reading documentation seem...
I want to use cli tool for file comparison and need line-number before output line with which help I could jump to line difference, because I use tool which understands where to jump, if the line begins like this :line-number: regular line contents So I tried diff, and reading documentation seems like it might be possible: -D, --ifdef=NAME output merged file with `#ifdef NAME' diffs --GTYPE-group-format=GFMT format GTYPE input groups with GFMT --line-format=LFMT format all input lines with LFMT --LTYPE-line-format=LFMT format LTYPE input lines with LFMT These format options provide fine-grained control over the output of diff, generalizing -D/--ifdef. LTYPE is old', new', or unchanged'. GTYPE is LTYPE or changed'. GFMT (only) may contain: % lines from FILE2 %= lines common to FILE1 and FILE2 %[-][WIDTH][.[PREC]]{doxX}LETTER printf-style spec for LETTER LETTERs are as follows for new group, lower case for old group: F first line number L last line number N number of lines = L-F+1 E F-1 M L+1 %(A=B?T:E) if A equals B then T else E LFMT (only) may contain: %L contents of line %l contents of line, excluding any trailing newline %[-][WIDTH][.[PREC]]{doxX}n printf-style spec for input line number Both GFMT and LFMT may contain: %% % %c'C' the single character C %c'\OOO' the character with octal code OOO C the character C (other characters represent themselves) but there is no example or explanation about this complicated switch. Is it possible to get such output from diff? If so how?
zetah (2117 rep)
Mar 23, 2012, 10:38 PM • Last activity: Jan 16, 2025, 12:59 PM
51 votes
4 answers
22549 views
How do I pipe colored diff output to less?
I've been using git diff, which produces colored output. However, I now find I need to use ordinary diff for something, and it's producing a lot of output that is hard to read because of the lack of colors. How do I make diff produce a readable, colored output? Ideally while piping it to less, for e...
I've been using git diff, which produces colored output. However, I now find I need to use ordinary diff for something, and it's producing a lot of output that is hard to read because of the lack of colors. How do I make diff produce a readable, colored output? Ideally while piping it to less, for easy review of large files.
Benubird (6082 rep)
May 17, 2013, 04:26 PM • Last activity: Jan 9, 2025, 04:21 PM
2 votes
0 answers
307 views
Why do two directories differ after syncing with rsync?
I copied some directories on one removable drive to another removable one using rsync. The command used was: rsync -avPS rsync doesn't report any errors. After syncing I used diff on some random directories to check for differences or errors. The synced files are apparently identical (checked with a...
I copied some directories on one removable drive to another removable one using rsync. The command used was: rsync -avPS rsync doesn't report any errors. After syncing I used diff on some random directories to check for differences or errors. The synced files are apparently identical (checked with a simple ls -lh), nevertheless diff reports that they differ. diff The files are all RAW photo files and their sidecar files (text files). What could be the reason for the file differences? ### Edit The filesystems are (partial output from fdisk -l): - SRC:
/dev/sdb1            2048  1953521663   976759808    7  HPFS/NTFS/exFAT
- DST:
/dev/sdd1              63  1953520064   976760001    7  HPFS/NTFS/exFAT
This is the output of ls -l: - SRC:
-rw------- 1 twan twan 177556828 Apr 19 12:38 20160403101701-07.tiff
- DST:
-rw------- 1 twan twan 177556828 Apr 19 12:38 20160403101701-07.tiff
Here is the output of diff for this specific file:
Binary files /media/twan/GrafxMedia/photos/hdr/20160403101701-07.tiff and ./20160403101701-07.tiff differ
twan163 (5770 rep)
Jun 2, 2016, 07:58 AM • Last activity: Dec 4, 2024, 02:08 PM
4 votes
1 answers
332 views
How to limit "diff -r" to a certain depth?
How can I compare sub-directories in two directories down to a certain directory depth? Command `diff` doesn't have an option to specify depth, and `-r` option takes you all the way down to the files, which I want to avoid. Basically, I would like to do this: cd dir1 for d in */*/*; do test -d dir2/...
How can I compare sub-directories in two directories down to a certain directory depth? Command diff doesn't have an option to specify depth, and -r option takes you all the way down to the files, which I want to avoid. Basically, I would like to do this: cd dir1 for d in */*/*; do test -d dir2/$d || echo $d; done but more elegantly with a single command line and no loop.
Renat (141 rep)
Oct 11, 2022, 08:31 AM • Last activity: Nov 17, 2024, 09:35 AM
43 votes
10 answers
26046 views
Compare directories but not content of files
With ``` diff -r ``` I can do this task, however it takes so long because `diff` checks file's content. I want something that determine that two files are the same regarding of their size, last modified, etc. But no checking bit by bit the file (for example a video takes sooo long). Is there any oth...
With
diff -r
I can do this task, however it takes so long because diff checks file's content. I want something that determine that two files are the same regarding of their size, last modified, etc. But no checking bit by bit the file (for example a video takes sooo long). Is there any other way?
user27807
Dec 24, 2012, 06:06 PM • Last activity: Nov 6, 2024, 02:16 AM
20 votes
4 answers
31882 views
What does 'patch unexpectedly ends in middle of line' mean?
This is the output of my patch command: Hunk #11 merged at 4184,4190. Hunk #12 merged at 4444. Hunk #13 merged at 4944. Hunk #14 NOT MERGED at 5106-5116. Hunk #15 merged at 5290. Hunk #16 merged at 5448. patch unexpectedly ends in middle of line Hunk #17 merged at 5608,5611. The command was patch -d...
This is the output of my patch command: Hunk #11 merged at 4184,4190. Hunk #12 merged at 4444. Hunk #13 merged at 4944. Hunk #14 NOT MERGED at 5106-5116. Hunk #15 merged at 5290. Hunk #16 merged at 5448. patch unexpectedly ends in middle of line Hunk #17 merged at 5608,5611. The command was patch -d ~/SOME_DIR -p1 --merge --verbose -u The patch was produced using git: git --git-dir ~/SOME_DIR/.git diff -U8 bb1ee538982957b421a4c0e78f30428e73c9a072 HEAD -- malloc.c What does patch unexpectedly ends in middle of line mean, and is it a problem? Is it referring to hunk 16 or 17? What can I look for in the patch file to figure out what's causing this?
Paul Biggar (302 rep)
Aug 30, 2010, 03:07 PM • Last activity: Nov 2, 2024, 06:54 PM
Showing page 1 of 20 total questions