Unix & Linux Stack Exchange
Q&A for users of Linux, FreeBSD and other Unix-like operating systems
Latest Questions
3
votes
3
answers
649
views
How to remove 'newline' from 'here string'
The 'here string' (` | md5sum` or `printf ... | md5sum` instead. But my question specifically pertains to the 'here string'.
The 'here string' (
| md5sum
or printf ... | md5sum
instead. But my question specifically pertains to the 'here string'.
Seamus
(3772 rep)
Feb 7, 2025, 05:02 AM
• Last activity: Feb 9, 2025, 12:51 AM
-1
votes
2
answers
647
views
What is the end of here string?
It seemed to me that the end of a here string is newline. I realize I am wrong: $ cat <<< hello world cat: world: No such file or directory What can signify the end of a here string?
It seemed to me that the end of a here string is newline. I realize I am wrong:
$ cat <<< hello world
cat: world: No such file or directory
What can signify the end of a here string?
Tim
(106420 rep)
Nov 22, 2018, 09:28 PM
• Last activity: Feb 8, 2025, 11:51 PM
2
votes
1
answers
147
views
bash: what's the difference between '< file' and taking input from a here-string which contains the file?
I have a variable `foo_var` which contains ASCII text (base64 text, with some additional text which includes dashes, spaces, and underscores). I write this var to a file as follows: cat foo_file I have a program which reads from `stdin` (`openssl`). This invocation works, with an exit code of 0: $ o...
I have a variable
foo_var
which contains ASCII text (base64 text, with some additional text which includes dashes, spaces, and underscores). I write this var to a file as follows:
cat foo_file
I have a program which reads from stdin
(openssl
). This invocation works, with an exit code of 0:
$ openssl rsa -passin stdin -pubout < foo_file
but both these fail, with a message saying that stdin
can't be read, and an exit code of 1:
$ openssl rsa -passin stdin -pubout <<<"$foo_var"
$ echo "$foo_var" | openssl -passin stdin -pubout
What's the difference between these cases? The obvious answer is that parameter expansion occurred in the failing case, but this can't be right, because the same parameter expansion would have happened when foo_file
was created. I'm on bash 5.1.16.
*EDIT*
Added the args, as requested. However, there's a complication - the args ask for both a password and a private key on stdin. This relies on what seems to be an undocumented openssl feature, which is that a single input stream can be used for both, but the password must be on the first line, followed by the key. So, it's not trivial to repeat. If anyone really wants a go I'll post a working password/key file.
*EDIT2*
I've done some debug by running under strace. For both the working and failing cases, there's some (essentially identical) initial setup, and then a read
request is made on stdin, for 4K bytes. This returns 1887 (the number of bytes actually available), and therefore advances the file position to 1887. In both cases, openssl then attempts an lseek
(lseek 0, -1875, SEEK_CUR)
). For the failing case, this fails with an ESPIPE
, as you might expect, and the program terminates soon after. For the passing case (redirection from an actual file) lseek
succeeds, returning a new file position of 12.
Ok, so nobody expects stdin
to be seekable. But bash sets up everything in both cases, and in both cases openssl
sees only stdin
. It can seek in one case and not the other. But openssl
knows that the input is stdin
, because that's a command-line parameter, so why does it seek
at all??
Conclusion? (A) Input redirection from a file and a here string are *not* equivalent, because Linux somehow makes seeking possible in the first case, or (B) openssl has messed up by attempting the lseek, and is taking advantage of a bug/feature? In any event, it should never have attempted the lseek, because it already knew what the data was.
QF0
(391 rep)
May 24, 2024, 01:42 PM
• Last activity: May 25, 2024, 01:26 PM
5
votes
3
answers
2241
views
"Here Document" as a Single Line Command?
I have this command to run a checksum in some software I want to install: echo "85762db0edc00ce19a2cd5496d1627903e6198ad850bbbdefb2ceaa46bd20cbd install.sh" | sha256sum -c But I would like to use it with the `sha256sum` command *first*. So we have that using heredoc: sha256sum -c << EOF 85762db0edc0...
I have this command to run a checksum in some software I want to install:
echo "85762db0edc00ce19a2cd5496d1627903e6198ad850bbbdefb2ceaa46bd20cbd install.sh" | sha256sum -c
But I would like to use it with the
sha256sum
command *first*. So we have that using heredoc:
sha256sum -c << EOF
85762db0edc00ce19a2cd5496d1627903e6198ad850bbbdefb2ceaa46bd20cbd install.sh
EOF
But now I would like this to be all on one line... So I tried:
sha256sum -c << EOF ; 85762db0edc00ce19a2cd5496d1627903e6198ad850bbbdefb2ceaa46bd20cbd install.sh ; EOF
And it doesn't work. But _shouldn't_ it work if the semicolon ;
means "we're going to run another command"?
Can we get it in that order, but written a different way to work on one line? If not, why can't we tell it what to do on each multiple line, but on one line?
Cnnewb
(51 rep)
Mar 29, 2024, 07:38 PM
• Last activity: Apr 1, 2024, 08:34 AM
-1
votes
2
answers
415
views
Difference in bash 3 between pipe and here-string
If I run: ``` IFS=':' cat <<< $(echo $PATH) ``` then the ":" are used for word splitting and the output contains spaces instead of ":", but if I run: ``` echo $PATH | IFS=':' cat ``` they are not. My understanding of pipes `a | b` is that they connect fd 1 of `a` to fd 0 of `b`. And my understanding...
If I run:
IFS=':' cat <<< $(echo $PATH)
then the ":" are used for word splitting and the output contains spaces instead of ":", but if I run:
echo $PATH | IFS=':' cat
they are not.
My understanding of pipes a | b
is that they connect fd 1 of a
to fd 0 of b
. And my understanding of herestrings a <<< string
is that they write the string directly to fd 0 of a
. In either case it seems like the same thing should be written to fd 0 of cat.
That suggests that there should be no difference between the two commands, but clearly there must be something I've misunderstood because this shows a diff:
diff <(echo $PATH | IFS=':' cat) <(IFS=':' cat <<< $(echo $PATH))
I am not trying to accomplish something and asking for some text I can cut and paste to do it. I have described a situation that contradicts my understanding and have asked for an explanation as to what I have misunderstood.
Can someone explain why these two commands produce different output?
----
I just tried this on bash 4 and do not get a difference there. That's why I asked "What version of Bash are you on"... So, I've edited the question to limit it to bash 3. I am not asking about bash 4.
Again, the question is not "can someone please give me something I can cut and paste to do a thing without thinking about it". The question is "what is the difference between these two things".
Allen
(206 rep)
Aug 4, 2023, 07:13 PM
• Last activity: Aug 7, 2023, 08:13 AM
-2
votes
2
answers
661
views
How to use positional parameter in docker exec command with here strings (<<<)?
Here is what I am trying to do: docker exec -it $1 /bin/bash <<< $CONTAINER_ID But this doesn't work: Output: "docker exec" requires at least 2 arguments. It seems that <<< completely replaces STDIN ignoring $1. Is it possible to replace only one argument with <<<? I know about aliases, but it doesn...
Here is what I am trying to do:
docker exec -it $1 /bin/bash <<< $CONTAINER_ID
But this doesn't work:
Output: "docker exec" requires at least 2 arguments.
It seems that <<< completely replaces STDIN ignoring $1. Is it possible to replace only one argument with << I know about aliases, but it doesn't suit me. I need to be able to use only standard tools to pass to my colleague a command in which he will change only the last variable, which will replace one or more arguments in the center of the command.
Anton Kuznetsov
(1 rep)
May 11, 2023, 06:41 AM
• Last activity: May 12, 2023, 09:22 AM
7
votes
3
answers
1925
views
How can I use the bash <<< operator with diff?
Please, observe: ```lang-shellsession $ diff b ``` I also know `<<<` works fine in general: ```lang-shellsession $ cat <<<a a ``` So what is the right way to call it with `diff`?
mark
(379 rep)
Apr 23, 2023, 08:48 PM
• Last activity: Apr 26, 2023, 01:33 PM
1
votes
1
answers
221
views
How can I concat results in the here document?
I have this script: ``` while read $item; do # Some bash logic here done <<< "$({ cat /path/to/some/file; echo; })" ``` Now I want to also use `find` to find the name of some directories, and pass that alongside the `cat` to the `while` loop. In other words, I want to: - `cat` a file - `find` some d...
I have this script:
while read $item;
do
# Some bash logic here
done <<< "$({ cat /path/to/some/file; echo; })"
Now I want to also use find
to find the name of some directories, and pass that alongside the cat
to the while
loop.
In other words, I want to:
- cat
a file
- find
some directories
- Concatenate the results
- Feed them as here-document to the while
loop
I'm stuck at this point.
I though of duplicating the while
logic, which is not a good method. I also thought of reusing the logic inside the while
loop using functions.
However, concatenating stuff in here-document might seem to be the correct way to do it.
Saeed Neamati
(841 rep)
Feb 24, 2023, 03:14 AM
• Last activity: Feb 25, 2023, 01:07 PM
0
votes
1
answers
58
views
When would someone choose to use a limit string (<<-) instead of format here strings the normal way with <<<?
```The - option to mark a here document limit string (<<-LimitString) will suppress leading tabs (but not spaces) in the output.``` What I'm wondering is why someone would want to remove the leading tabs when they are processing commands with the "here" method. Is it about readability of output, or...
- option to mark a here document limit string (<<-LimitString) will suppress leading tabs (but not spaces) in the output.
What I'm wondering is why someone would want to remove the leading tabs when they are processing commands with the "here" method. Is it about readability of output, or cleaning up output if we want to pass it to something else?
thinksinbinary
(217 rep)
Feb 12, 2023, 07:19 PM
• Last activity: Feb 12, 2023, 08:42 PM
1
votes
2
answers
436
views
How to make a script fail when there is an error in here string?
I have a script similar to this: ```bash #!/bin/bash set -euo pipefail IFS=$'\n\t' while read -r l; do echo "${l}" done <<< "$(cat input.txt)" echo Success ``` The command ```cat input.txt``` is just an example here to simplify my question. I expected that if ```input.txt``` does not exist, the scri...
I have a script similar to this:
#!/bin/bash
set -euo pipefail
IFS=$'\n\t'
while read -r l; do
echo "${l}"
done <<< "$(cat input.txt)"
echo Success
The command input.txt
is just an example here to simplify my question.
I expected that if .txt
does not exist, the script would exit immediately thanks to -euo pipefail
. But it's not the case, instead the script ends successfully with the following output:
cat: input.txt: No such file or directory
Success
Is there a way to make the script fail as I expected?
Thomas Leplus
(155 rep)
Dec 16, 2022, 02:04 AM
• Last activity: Dec 18, 2022, 08:40 AM
2
votes
0
answers
121
views
Why does `rm writeprotectedfile <<< ""` delete the file?
The default behaviour of the GNU `rm` command when asked to delete write-protected files is to interactively ask whether the user wants to delete each of the files. This is quite inconvenient in many situations. A crude general way to supply an interactive command invocation with _no meaningful inpu...
The default behaviour of the GNU
(Ubuntu 18.04, GNU Bash 4.4.20, coreutils 8.28) What's going on here, and does it also have ramifications for similar situations?
rm
command when asked to delete write-protected files is to interactively ask whether the user wants to delete each of the files. This is quite inconvenient in many situations.
A crude general way to supply an interactive command invocation with _no meaningful input_ is to use a here-string. I expected that in this case,
touch wpf && chmod -w wpf
rm wpf enter or ctrl+d upon the prompt.
To my surprise however, it actually results in the file being deleted. (Ubuntu 18.04, GNU Bash 4.4.20, coreutils 8.28) What's going on here, and does it also have ramifications for similar situations?
leftaroundabout
(662 rep)
Sep 12, 2022, 01:58 PM
• Last activity: Sep 12, 2022, 02:05 PM
3
votes
2
answers
13515
views
How to use mapfile/readarray
I have some code similar to this: while read -r col1 col2 col3 col4 col5 col6 col7 col8 TRASH; do echo -e "${col1}\n${col2}\n${col3}\n${col4}\n${col5}\n${col6}\n" done echo -e "${col1}\n${col2}\n${col3}\n${col4}\n${col5}\n${col6}\n" > done<<<"${TEST_ARR[@]}" drwxr-xr-x@ 38 wheel 1.2K Dec 7 String re...
I have some code similar to this:
while read -r col1 col2 col3 col4 col5 col6 col7 col8 TRASH; do
echo -e "${col1}\n${col2}\n${col3}\n${col4}\n${col5}\n${col6}\n"
done echo -e "${col1}\n${col2}\n${col3}\n${col4}\n${col5}\n${col6}\n"
> done<<<"${TEST_ARR[@]}"
drwxr-xr-x@
38
wheel
1.2K
Dec
7
String redirect is clearly wrong in this case but I'm not sure how else I can redirect my array.
jesse_b
(41447 rep)
Jan 19, 2018, 10:30 PM
• Last activity: Jun 19, 2022, 03:38 PM
4
votes
3
answers
7954
views
Variable not interpreted with 'EOF'
This is my script: var="lalallalal" tee file.tex <<'EOF' text \\ text \\ $var EOF Need to use 'EOF' (with quotes) because I can not use double slash (`//`) otherwise. However, if I use the quotes, the `$var` variable is not expanded.
This is my script:
var="lalallalal"
tee file.tex <<'EOF'
text \\ text \\
$var
EOF
Need to use 'EOF' (with quotes) because I can not use double slash (
//
) otherwise.
However, if I use the quotes, the $var
variable is not expanded.
Marcus
(49 rep)
Jul 19, 2017, 04:20 PM
• Last activity: May 21, 2022, 12:59 PM
2
votes
1
answers
257
views
sed with here-string fails, but succeeds when echo output piped to sed
After updating `sed` to `4.4` version, `sed` doesn't replace spaces with commas in the output of `find` command given to it as `here-string`: ``` sed --version sed (GNU sed) 4.4 ls -l /tmp/test/ total 0 -rw-r--r-- 1 root root 0 Jan 9 17:25 a -rw-r--r-- 1 root root 0 Jan 9 17:25 b # NOT EXPECTED sed...
After updating
sed
to 4.4
version, sed
doesn't replace spaces with commas in the output of find
command given to it as here-string
:
sed --version
sed (GNU sed) 4.4
ls -l /tmp/test/
total 0
-rw-r--r-- 1 root root 0 Jan 9 17:25 a
-rw-r--r-- 1 root root 0 Jan 9 17:25 b
# NOT EXPECTED
sed "s: :,:g" <<< $(find /tmp/test/ -type f)
/tmp/test/b
/tmp/test/a
There are no issue in sed
4.2
sed --version
sed (GNU sed) 4.2.2
ls -l /tmp/test/
total 0
-rw-r--r-- 1 root root 0 Jan 9 17:25 a
-rw-r--r-- 1 root root 0 Jan 9 17:25 b
# as expected
sed "s: :,:g" <<< $(find /tmp/test/ -type f)
/tmp/test/a,/tmp/test/b
As a workaround, storing the result in variable and using echo
helps:
a=$(find /tmp/test/ -type f)
echo $a | sed "s: :,:g"
/tmp/test/b,/tmp/test/a
How to achieve the same output in sed
4.4 using here-string
?
**Update**
version of bash changed as well between the two systems:
bash --version
GNU bash, version 4.4.20
old version
bash --version
GNU bash, version 4.3.48
rokpoto.com
(419 rep)
Jan 9, 2022, 05:34 PM
• Last activity: Jan 9, 2022, 05:57 PM
0
votes
2
answers
428
views
Ubuntu - How do I pass JSON string to herestring?
Update ============== I noticed that I am able to pass JSON with this command command -j /dev/stdin <<< '{"key":"value"}' However, it does not work if I call it through **SSH**. ssh {target} 'command -j /dev/stdin <<< '{"key":"value"}'' Looks like it send through as a string and not JSON? Anyone hav...
Update
==============
I noticed that I am able to pass JSON with this command
command -j /dev/stdin <<< '{"key":"value"}'
However, it does not work if I call it through **SSH**.
ssh {target} 'command -j /dev/stdin <<< '{"key":"value"}''
Looks like it send through as a string and not JSON? Anyone have any idea why?
___
I have a command where I need to pass a JSON string to the options but for some reasons I need it to be passed using
herestring
to /dev/stdin
.
**Example**
command -j /dev/stdin <<< '{"key":"value"}'
Norman
(139 rep)
Jul 26, 2019, 07:11 AM
• Last activity: Jun 28, 2021, 09:53 AM
6
votes
2
answers
1120
views
What is more efficient or recommended for reading output of a command into variables in Bash?
If you want to read the single line output of a system command into Bash shell variables, you have at least two options, as in the examples below: 1. `IFS=: read user x1 uid gid x2 home shell <<<$(grep :root: /etc/passwd | head -n1)` and 2. `IFS=: read user x1 uid gid x2 home shell < <(grep :root: /...
If you want to read the single line output of a system command into Bash shell variables, you have at least two options, as in the examples below:
1.
IFS=: read user x1 uid gid x2 home shell <<<$(grep :root: /etc/passwd | head -n1)
and
2. IFS=: read user x1 uid gid x2 home shell < <(grep :root: /etc/passwd | head -n1)
Is there any difference between these two? What is more efficient or recommended?
---
Please note that, reading the /etc/passwd
file is just for making an example. The focus of my question is on _here strings_ vs. _process substitution_.
FedKad
(618 rep)
Jun 13, 2021, 08:12 AM
• Last activity: Jun 14, 2021, 01:37 PM
3
votes
1
answers
494
views
backtick or Here string or read is not working as expected in RHEL 8
I am trying to redirect the output of a python script as an input into an interactive shell script. **test.py** ``` print('Hello') print('world') ``` Say test.py is as above prints *"Hello world"* which is feed to two variables using **Here string** redirection as below ***Interactive script*** : ``...
I am trying to redirect the output of a python script as an input into an interactive shell script.
**test.py**
print('Hello')
print('world')
Say test.py is as above prints *"Hello world"* which is feed to two variables using **Here string** redirection as below
***Interactive script*** : `` read a b read a b echo $a $b
Hello
tmp> cat /etc/redhat-release
Red Hat Enterprise Linux release 8.3 (Ootpa)
**variable b is empty** in rhel 8
**Rhel 7:**
tmp> read a b echo $a $b
Hello world
tmp> cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)
while the **read & Here string** works fine in both cases as below
tmp> read a b echo $a $b
Hello world
```
ragul rangarajan
(143 rep)
Apr 2, 2021, 06:16 AM
• Last activity: Apr 2, 2021, 07:32 AM
6
votes
2
answers
1172
views
Are Here-Strings available in ksh88?
I wanted to extract 3 words from a variable in 3 different variables in ksh script. I used this line on ksh93 and it worked fine: `read A B C <<< $line` Got this error for the above command while running it on ksh88: `syntax error at line ## : '<' unexpected` How to perform the same action on ksh88?
I wanted to extract 3 words from a variable in 3 different variables in ksh script. I used this line on ksh93 and it worked fine:
read A B C <<< $line
Got this error for the above command while running it on ksh88:
syntax error at line ## : '<' unexpected
How to perform the same action on ksh88?
Pratik Mayekar
(639 rep)
Jun 18, 2018, 12:14 PM
• Last activity: Mar 16, 2021, 05:53 AM
10
votes
4
answers
4611
views
Why does cut fail with bash and not zsh?
I create a file with tab-delimited fields. echo foo$'\t'bar$'\t'baz$'\n'foo$'\t'bar$'\t'baz > input I have the following script named `zsh.sh` #!/usr/bin/env zsh while read line; do <<<$line cut -f 2 done < "$1" I test it. $ ./zsh.sh input bar bar This works fine. However, when I change the first li...
I create a file with tab-delimited fields.
echo foo$'\t'bar$'\t'baz$'\n'foo$'\t'bar$'\t'baz > input
I have the following script named
zsh.sh
#!/usr/bin/env zsh
while read line; do
<<<$line cut -f 2
done < "$1"
I test it.
$ ./zsh.sh input
bar
bar
This works fine. However, when I change the first line to invoke bash
instead, it fails.
$ ./bash.sh input
foo bar baz
foo bar baz
Why does this fail with bash
and work with zsh
?
## Additional troubleshooting
* Using direct paths in the shebang instead of env
produces the same behaviour.
* Piping with echo
instead of using the here-string <<<$line
also produces the same behaviour. i.e. echo $line | cut -f 2
.
* Using awk
instead of cut
*works* for both shells. i.e. <<<$line awk '{print $2}'
.
Sparhawk
(20499 rep)
Jun 10, 2016, 11:21 AM
• Last activity: Jan 4, 2021, 10:46 AM
1
votes
2
answers
581
views
sshpass Solaris one liner passing a command
I am trying to pass one liner sshpass command to Solaris box but won't work. those are the options I tried: sshpass -p "passw" ssh "root@135.102.22.0" -o StrictHostKeyChecking=no `at 11:04 Dec 10 <<< touch /tmp/test_file` sshpass -p "passw" ssh "root@135.102.22.0" -o StrictHostKeyChecking=no 'at 11:...
I am trying to pass one liner sshpass command to Solaris box but won't work. those are the options I tried:
sshpass -p "passw" ssh "root@135.102.22.0" -o StrictHostKeyChecking=no "at 13:19 Dec 10 <
and that gives me error: at: bad time specification
at 11:04 Dec 10 <<< touch /tmp/test_file
sshpass -p "passw" ssh "root@135.102.22.0" -o StrictHostKeyChecking=no 'at 11:04 Dec 10 <<< "touch /tmp/test_file"'
sshpass -p "passw" ssh "root@135.102.22.0" -o StrictHostKeyChecking=no "at 11:04 Dec 10 <<< "touch /tmp/test_file""
sshpass -p "passw" ssh "root@135.102.22.0" -o StrictHostKeyChecking=no "at 11:04 Dec 10 <<< \"touch /tmp/test_file\""
also tried this variations:
"at 11:04 Dec 10 <<<\"touch /tmp/test_file\""
"at 11:04 Dec 10 <<
dwt.bar
(145 rep)
Dec 10, 2020, 04:49 PM
• Last activity: Dec 13, 2020, 10:04 PM
Showing page 1 of 20 total questions