Sample Header Ad - 728x90

Handling Bash script with CRLF (carriage return) in Linux as in MSYS2?

13 votes
5 answers
26971 views
Let's say I have the following trivial script, tmp.sh:
echo "testing"
stat .
echo "testing again"
Trivial as it is, it has \r\n (that is, CRLF, that is carriage return+line feed) as line endings. Since the webpage will not preserve the line endings, here is a hexdump:
$ hexdump -C tmp.sh 
00000000  65 63 68 6f 20 22 74 65  73 74 69 6e 67 22 0d 0a  |echo "testing"..|
00000010  73 74 61 74 20 2e 0d 0a  65 63 68 6f 20 22 74 65  |stat ...echo "te|
00000020  73 74 69 6e 67 20 61 67  61 69 6e 22 0d 0a        |sting again"..|
0000002e
Now, it has CRLF line endings, because the script was started and developed on Windows, under MSYS2. So, when I run it on Windows 10 in MSYS2, I get the expected:
$ bash tmp.sh
testing
  File: .
  Size: 0               Blocks: 40         IO Block: 65536  directory
Device: 8e8b98b6h/2391513270d   Inode: 281474976761067  Links: 1
Access: (0755/drwxr-xr-x)  Uid: (197609/      USER)   Gid: (197121/    None)
Access: 2020-04-03 10:42:53.210292000 +0200
Modify: 2020-04-03 10:42:53.210292000 +0200
Change: 2020-04-03 10:42:53.210292000 +0200
 Birth: 2019-02-07 13:22:11.496069300 +0100
testing again
However, if I copy this script to an Ubuntu 18.04 machine, and run it there, I get something else:
$ bash tmp.sh
testing
stat: cannot stat '.'$'\r': No such file or directory
testing again
In other scripts with the same line endings, I have also gotten this error in Ubuntu bash: line 6: $'\r': command not found ... likely from an empty line. So, clearly, something in Ubuntu chokes on the carriage returns. I have seen https://unix.stackexchange.com/questions/355559/bash-and-carriage-return-behavior : > it doesn’t have anything to do with Bash: \r and \n are interpreted by the terminal, not by Bash ... however, I guess that is only for stuff typed verbatim on the command line; here the \r and \n are already typed in the script itself, so it must be that Bash interprets the \r here. Here is the version of Bash in Ubuntu:
$ bash --version
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
... and here the version of Bash in MSYS2:
$ bash --version
GNU bash, version 4.4.23(2)-release (x86_64-pc-msys)
(they don't seem all that much apart ...) Anyways, my question is - is there a way to persuade Bash on Ubuntu/Linux to ignore the \r, rather than trying to interpret it as a (so to speak) "printable character" (in this case, meaning a character that could be a part of a valid command, which bash interprets as such)? EDIT: *without* having to convert the script itself (so it remains the same, with CRLF line endings, if it is checked in that way, say, in git) EDIT2: I would prefer it this way, because other people I work with might reopen the script in Windows text editor, potentially reintroduce \r\n again into the script and commit it; and then we might end up with an endless stream of commits which might be nothing else than conversions of \r\n to \n polluting the repository. EDIT2: @Kusalananda in comments mentioned dos2unix (sudo apt install dos2unix); note that just writing this:
$ dos2unix tmp.sh 
dos2unix: converting file tmp.sh to Unix format...
... will convert the file in-place; to have it output to stdout, one must setup stdin redirection:
$ dos2unix 
... and then, in principle, one could run this on Ubuntu, which seems to work in this case:
$ dos2unix 
However, - aside from the slightly messy command to remember - this also changes bash semantics, as stdin is no longer a terminal; this may have worked with this trivial example, but see e.g. https://stackoverflow.com/questions/23257247/pipe-a-script-into-bash for example of bigger problems.
Asked by sdaau (7098 rep)
Apr 3, 2020, 09:01 AM
Last activity: Nov 17, 2023, 10:15 PM