Sample Header Ad - 728x90

Unix & Linux Stack Exchange

Q&A for users of Linux, FreeBSD and other Unix-like operating systems

Latest Questions

3 votes
1 answers
200 views
Why SIGTSTP (^Z/Ctrl-Z/suspend) doesn't work when process blocked redirecting write into a FIFO/pipe not open for reading yet? And what to do then?
Sometimes, a command is stalled attempting to write to a FIFO/pipe that no other process is currently reading from, but typing Ctrl - Z to put the process to the background by sending it the SIGTSTP signal (i.e. the *suspend* non-printing control character `^Z`) does work. Example: $ mkfifo p $ ll p...
Sometimes, a command is stalled attempting to write to a FIFO/pipe that no other process is currently reading from, but typing Ctrl-Z to put the process to the background by sending it the SIGTSTP signal (i.e. the *suspend* non-printing control character ^Z) does work. Example: $ mkfifo p $ ll p prw-rw-r-- 1 me me 0 Jul 30 16:27 p| $ echo "Go!" >p # Here & has been forgotten after the >p redirection (stalled) [Ctrl-Z] ^Z # Pressing Ctrl-Z just prints "^Z" on the terminal, nothing else happens [Ctrl-D] # Attempting to send the EOF character (nothing) # Doesn't print "^D" and does nothing and this is the behavior even though SIGTSTP (or susp) is reported to be attached to ^Z: $ stty -a speed 38400 baud; rows 24; columns 91; line = 0; intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = ; eol2 = ; swtch = ; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0; -parenb -parodd -cmspar cs8 -hupcl -cstopb cread -clocal -crtscts -ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl -ixon -ixoff -iuclc -ixany -imaxbel iutf8 opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0 isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke -flusho -extproc How comes that in this case the terminal does not catch and transmit to the process the SIGTSTP signal generated by the Ctrl-Z? Is it because redirecting with > to a FIFO/pipe puts the terminal in raw mode (cf. The TTY demystified )? But if so, why does "^Z" get printed on the screen, and why Ctrl-D (EOF) doesn't print anything and yet doesn't have any effect either? Incidentally, is there an alternative way to send the stalled process to the background in such cases (instead of just terminate the process with Ctrl-C (i.e. ^C / SIGINT)?
The Quark (402 rep)
Aug 5, 2025, 08:47 AM • Last activity: Aug 5, 2025, 12:02 PM
0 votes
1 answers
2211 views
Subshell and process substitution
Apologies if this is a basic question - I'm stuck trying to solve a larger problem, and it's come down to how a shell script is invoked - directly (`shellScript.sh`) or using `sh shellScript.sh`. Here's a model for the problem: When I execute on bash: cat <(echo 'Hello') I see the output Hello But w...
Apologies if this is a basic question - I'm stuck trying to solve a larger problem, and it's come down to how a shell script is invoked - directly (shellScript.sh) or using sh shellScript.sh. Here's a model for the problem: When I execute on bash: cat <(echo 'Hello') I see the output Hello But when I use: sh -c "cat <(echo 'Hello')" I see errors: sh: -c: line 0: syntax error near unexpected token `(' sh: -c: line 0: `cat <(echo 'Hello')' I've tried escaping the <, ( and ) in various combinations, but I don't see the output anywhere. What am I missing here? My actual problem is that I'm passing a <() as an input argument to a python script within a shell script, and while it works fine when I invoke the shell script using just the name, if I use sh to invoke it, I get errors similar to what I've shown above. Thank you!
Ram RS (111 rep)
May 9, 2017, 08:16 PM • Last activity: Jul 30, 2025, 06:24 AM
2 votes
1 answers
631 views
1-liner process substitution for ksh
I have this command, which works fine for `bash`: `sh <(curl -sSl https://appi.sh/launch)` (launches [this interactive script](https://github.com/Appish/appi.sh/blob/master/launch.sh)) I'd like to get this 1-liner working for `ksh`, too. Currently, it errors with: ``` syntax error: `(' unexpected` `...
I have this command, which works fine for bash: sh <(curl -sSl https://appi.sh/launch) (launches [this interactive script](https://github.com/Appish/appi.sh/blob/master/launch.sh)) I'd like to get this 1-liner working for ksh, too. Currently, it errors with:
syntax error: (' unexpected
Running the script from ksh as sh launch.sh works fine, so I'm hoping there's a way to have a singular 1-liner that will work in both bash/ksh without exponentially increasing the length of the command.
ljs.dev (2147 rep)
Jun 17, 2020, 05:58 PM • Last activity: Jul 22, 2025, 05:09 PM
0 votes
0 answers
162 views
Process substitution doesn't work with diff and node
I'm trying to compare the output of my `fizzbuzz.js` program: for (let i = 1; i <= 100; i++) { let out = ""; if (i % 3 == 0) out += "Fizz"; if (i % 5 == 0) out += "Buzz"; if (i % 3 && i % 5) out = i; console.log(out); } with the contents of a `expected-output.txt` file which contains: 1 2 Fizz 4 Buz...
I'm trying to compare the output of my fizzbuzz.js program: for (let i = 1; i <= 100; i++) { let out = ""; if (i % 3 == 0) out += "Fizz"; if (i % 5 == 0) out += "Buzz"; if (i % 3 && i % 5) out = i; console.log(out); } with the contents of a expected-output.txt file which contains: 1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 [...] I can't get process substitution to work: diff -u expected-output.txt <(node fizzbuzz.js) seems to hang indefinitely. What could be causing this? My environment is macOS Mojave, bash 3.2.57, node v12.5.0. EDIT: this seems to be a macOS issue. Everything works fine on a Ubuntu 18.04 machine with newer bash 4.4.19
Chef Tony (465 rep)
Jul 14, 2019, 06:09 PM • Last activity: Apr 12, 2025, 12:59 PM
41 votes
1 answers
9015 views
Why does BASH process substitution not work with some commands?
On occasion process substitution will not work as expected. Here is an example: Input: gcc <(echo 'int main(){return 0;}') Output: /dev/fd/63: file not recognized: Illegal seek collect2: error: ld returned 1 exit status Input: But it works as expected when used with a different command: grep main <(...
On occasion process substitution will not work as expected. Here is an example: Input: gcc <(echo 'int main(){return 0;}') Output: /dev/fd/63: file not recognized: Illegal seek collect2: error: ld returned 1 exit status Input: But it works as expected when used with a different command: grep main <(echo 'int main(){return 0;}') Output: int main(){return 0;} I have noticed similar failures with other commands (i.e. the command expecting the file from the process substitution can't use /dev/fd/63 or similar). This failure with gcc is just the most recent. Is there some general rule that I should be aware of to determine when process substitution will fail in this way and should not be used? I am using this BASH version on Ubuntu 12.04 (I've also seen this in arch and debian): GNU bash, version 4.3.11(1)-release (i686-pc-linux-gnu)
Lotney (577 rep)
Oct 25, 2014, 12:00 AM • Last activity: Feb 14, 2025, 11:47 AM
2 votes
3 answers
1566 views
Pipe output from process substitution to variable
I have very complicated set of commands: ``` command | ... | ... | tee >(grep -c '[^3]$') >(grep -c '[^35]$') 1>/dev/null ``` I don't want to have a temporary file to save up output, as it is pretty huge. I tried doing `>(grep -c '[^3]$' | read variable)` and `>(grep -c '[^3]$' | read variable2)` bu...
I have very complicated set of commands:
command | ... | ... | tee >(grep -c '[^3]$') >(grep -c '[^35]$') 1>/dev/null
I don't want to have a temporary file to save up output, as it is pretty huge. I tried doing >(grep -c '[^3]$' | read variable) and >(grep -c '[^3]$' | read variable2) but I guess bit doesn't work because of process substitution's sub-shell invocation. What can I do to pipe output directly to multiple variables? Is it even possible? Right now I have this workaround:
var=$(command ... | ... | ... | tee >(grep -c '[^3]$') >(grep -c '[^35]$') 1>/dev/null)
var1=$(tail -1 <<< $var)
var2=$(head -1 <<< $var)
but I think it is clumsy and doesn't look nice. I am aware I can grep into another file but I don't like it either. Thanks in advance.
Fedja (125 rep)
May 13, 2021, 02:23 PM • Last activity: Jan 14, 2025, 12:03 AM
22 votes
2 answers
6621 views
Difference between subshells and process substitution
In bash, I want to assign my current working directory to a variable. Using a subshell, I can do this. var=$(pwd) echo $var /home/user.name If I use process substitution like so: var=<(pwd) echo $var /dev/fd/63 I have understood that process substitution is primarily used when a program does not acc...
In bash, I want to assign my current working directory to a variable. Using a subshell, I can do this. var=$(pwd) echo $var /home/user.name If I use process substitution like so: var=<(pwd) echo $var /dev/fd/63 I have understood that process substitution is primarily used when a program does not accept STDIN. It is unclear to me what a process substitution exactly does and why it assigns /dev/fd/63 to var.
PejoPhylo (385 rep)
Sep 20, 2017, 08:12 AM • Last activity: Dec 20, 2024, 10:58 AM
14 votes
2 answers
2800 views
Why doesn't the process substitution <() work with ssh -F
I have some vagrant virtual machines. To log into them I issue the `vagrant ssh` command. I want to log into them using regular `ssh` command. The `vagrant ssh-config` outputs the suitable config file $ vagrant ssh-config Host default HostName 127.0.0.1 User vagrant Port 2201 UserKnownHostsFile /dev...
I have some vagrant virtual machines. To log into them I issue the vagrant ssh command. I want to log into them using regular ssh command. The vagrant ssh-config outputs the suitable config file $ vagrant ssh-config Host default HostName 127.0.0.1 User vagrant Port 2201 UserKnownHostsFile /dev/null StrictHostKeyChecking no PasswordAuthentication no IdentityFile /home/cbliard/.vagrant.d/insecure_private_key IdentitiesOnly yes LogLevel FATAL When outputing this config in a file and using with ssh -F, everything works fine: $ vagrant ssh-config > /tmp/config $ ssh -F /tmp/config default => logged successfully When using process substitution operator <(cmd) to prevent the creation of the temporary config file, it fails: $ ssh -F <(vagrant ssh-config) default Can't open user config file /proc/self/fd/11: No such file or directory Same error happens when using <(cat /tmp/config) $ ssh -F <(cat /tmp/config) default Can't open user config file /proc/self/fd/11: No such file or directory I am using zsh and I observe the same behavior with bash. What am I doing wrong here?
cbliard (402 rep)
Nov 20, 2013, 10:50 AM • Last activity: Sep 9, 2024, 06:07 AM
1 votes
2 answers
593 views
Replace a string in a script without modifying the file, and then to execute it
I have both BSD (default) and GNU Awk implementations installed on a Mac (the latter is envoked using `gawk` instead of just `awk`), and I have a shell script with contents like this: ``` none awk -v maxLen=72 ' . . . ' "$1" > "$2" ``` Now the tricky part. To work correctly, the script must use GNU...
I have both BSD (default) and GNU Awk implementations installed on a Mac (the latter is envoked using gawk instead of just awk), and I have a shell script with contents like this:
none
awk -v maxLen=72 '
    .
    .
    .
' "$1" > "$2"
Now the tricky part. To work correctly, the script must use GNU Awk. Which means I have at least two options: 1)To modify the first line of the script so that there will be gawk instead of awk, 2) To make a system-wide alias for GNU Awk so that it will be envoked using awk instead gawk. But I would prefer to avoid either option, and instead to execute the script using a wrapper. That is, instead of
none
./script.sh input.txt output.txt
I would prefer something like
none
sed -e 's/awk/gawk/' script.sh
and then to pass the output to Zsh somehow. I was told this is possible using Zsh process substitution feature, but how?
jsx97 (1347 rep)
Sep 2, 2024, 07:26 PM • Last activity: Sep 3, 2024, 05:58 AM
1 votes
1 answers
121 views
Use Zsh process substitution in place of mktemp
If you have Pandoc installed, the following script will retrieve its HTML template, write it into a temporary file, make some adjustments, and then pass this file as a parameter: ``` template=$(mktemp default.html5.XXXXXX) sed -z 's/ .* \s*//' > $template pandoc input.md -s --template $template -o o...
If you have Pandoc installed, the following script will retrieve its HTML template, write it into a temporary file, make some adjustments, and then pass this file as a parameter:
template=$(mktemp default.html5.XXXXXX)
sed -z 's/.*\s*//' > $template
pandoc input.md -s --template $template -o output.htm
rm $template
It works, but I learn how to use Zsh effectively, and for this reason I want to change it in such a way that creating a temporary file manually won't be required. As far as I know, Zsh process substitution feature creates temporary files automatically. For example:
${template::==(pandoc -D html)}
But how (and whether) it is possible to re-write the original script using this technique?
jsx97 (1347 rep)
Aug 6, 2024, 06:08 PM • Last activity: Aug 8, 2024, 07:03 AM
0 votes
0 answers
213 views
diff -qr command between local and remote directories
According to [this][1] process substitution can be used to do a SIMPLE diff between local and remote machine. However, I want to use the `-qr` option in the diff command, i.e., it must be recursive and brief (reports only files that differ). If I try to do ```` diff -qr <(ls -R ~/Documents) <(ssh my...
According to this process substitution can be used to do a SIMPLE diff between local and remote machine. However, I want to use the -qr option in the diff command, i.e., it must be recursive and brief (reports only files that differ). If I try to do
`
diff -qr <(ls -R ~/Documents) <(ssh myusername@192.168.1.150 ls -R /pathinmyremote/My_remote_Documents)
` I get Files /dev/fd/63 and /dev/fd/62 differ which is obviously not what I am looking for. There are suggestions of using rsync with the dry-run option, but that does not include the brief option. The reason for -qr is that I want to know 1) which files are in my local machine and missing in my remote backup, and 2) which files that have been changed/updated in my local machine with respect to the ones in the backup remote. **Update** The suggestion to use rsync -n -ai --delete ~/Documents myusername@192.168.1.150:/pathinmyremote/My_remote_Documents results in:
which is also not right (due to the existing files/directories)
Ajned (111 rep)
May 11, 2024, 07:03 AM • Last activity: May 11, 2024, 05:15 PM
1 votes
1 answers
906 views
how to keep stty sane after piping strace to vim?
this makes vim madness: $strace -o >(vim -; stty sane) file.out; stty sane I have typed `stty sane` in processed substitution as well as in next command but none of them did their job. Once i piped the strace command to vim, then the vim no longer behaves normaly (I know vim can only accepts `stdin`...
this makes vim madness: $strace -o >(vim -; stty sane) file.out; stty sane I have typed stty sane in processed substitution as well as in next command but none of them did their job. Once i piped the strace command to vim, then the vim no longer behaves normaly (I know vim can only accepts stdin and strace gives output to stderr, but then what is the -o flag for?) any help?
Herdsman (350 rep)
May 23, 2020, 02:55 PM • Last activity: Mar 28, 2024, 11:23 PM
0 votes
1 answers
74 views
ffmpeg inside while loop with find unintentionally alters iteration variable
Can someone please explain to me what is happening here? This is what I have reduced my situation down to: ```bash # make 20 test gifs out of the same source file. for i in {1..20}; do cp -p ../some-random-source-file.gif "${i}.gif"; done # grab, then process them. while read f; do echo "→ $f"; ffmp...
Can someone please explain to me what is happening here? This is what I have reduced my situation down to:
# make 20 test gifs out of the same source file.
for i in {1..20}; do cp -p ../some-random-source-file.gif "${i}.gif"; done

# grab, then process them.
while read f; do echo "→ $f"; ffmpeg -i "$f"  -y  -loglevel quiet  "same.mp4"; done < <(find .  -maxdepth 1  -type f  -iname "*.gif"  -printf "%f\n")
→ 11.gif
→ .gif
→ 9.gif
→ .gif
→ 14.gif
→ 9.gif
→ 0.gif
→ 13.gif
→ 7.gif
→ 5.gif
→ 2.gif
→ .gif
→ 3.gif
→ 0.gif
→ 16.gif
→ .gif
→ 8.gif
→ 8.gif
→ .gif
→ 4.gif
I am trying to process all gifs in a directory with mixed files (gifs and non-gifs). But for some reason, as soon as I add the ffmpeg step, the content of the $f variable is sometimes cut off at the beginning. --- Extra info: * I'm using process substition because I am also logging the files that didn't work. if ! ffmpeg …; then FAILED+=("$f"); fi * I'm also generating the new filename via string substitution ${f%.gif}.mp4, but … turns out: that part isn't even relevant for the problem to occur. * bash version 5.1.16 I understand that the process substitution [can cause timing issues](https://unix.stackexchange.com/a/190857/278372) – but then why does it randomly cut off the variable _at the beginning_ and not at the end? And how is anyone to use this construct if it is so unreliable? How else am I to do this?
WoodrowShigeru (261 rep)
Jan 20, 2024, 05:02 PM • Last activity: Jan 20, 2024, 05:27 PM
0 votes
0 answers
54 views
How to diff command outputs in vim
I tried `vim -d &1) &1)` to see the difference between the two videos' codecs etc., but vim doesn't show anything when it opens, just an empty split screen and this error > "/dev/fd/12" [Permission Denied] How can I conveniently use vim diff to compare two command outputs?
I tried vim -d &1) &1) to see the difference between the two videos' codecs etc., but vim doesn't show anything when it opens, just an empty split screen and this error > "/dev/fd/12" [Permission Denied] How can I conveniently use vim diff to compare two command outputs?
user172877
Jan 9, 2024, 03:25 PM
16 votes
4 answers
38714 views
How to pass to ssh the key stored in a shell environment variable?
Presumably `ssh` tries to access the `fd` several times, but the `fd` is destroyed after it has been read the very first time: # ssh -i <(echo $KEY) user@example.com Warning: Identity file /dev/fd/11 not accessible: Bad file descriptor. user@example.com: Permission denied (publickey). Is there any o...
Presumably ssh tries to access the fd several times, but the fd is destroyed after it has been read the very first time: # ssh -i <(echo $KEY) user@example.com Warning: Identity file /dev/fd/11 not accessible: Bad file descriptor. user@example.com: Permission denied (publickey). Is there any other way without writing/removing temporary file?
NarūnasK (2525 rep)
Apr 23, 2019, 11:56 AM • Last activity: Oct 20, 2023, 01:22 PM
20 votes
3 answers
6863 views
How to combine Bash's process substitution with HERE-document?
In Bash version 4.2.47(1)-release when I try to catenate formatted text that comes from a HERE-dcoument like so: cat <(fmt --width=10 <<FOOBAR (I want the surrounding parentheses to be part of the HERE-document) (Even the preceding unbalanced parenthesis should be part of it. FOOBAR ) # I want this...
In Bash version 4.2.47(1)-release when I try to catenate formatted text that comes from a HERE-dcoument like so: cat <(fmt --width=10 <<'FOOBAR', because I still want to have variables being substituted within it.
Tim Friske (2390 rep)
Jun 17, 2014, 12:49 AM • Last activity: Oct 17, 2023, 11:25 AM
0 votes
0 answers
110 views
docker build inline command fails with process substitution part
I am wondering why this docker build command fails: docker build -t inline_img -f ERROR [internal] load build definition from 11 0.0s => => transferring dockerfile: 55B 0.0s => CANCELED [internal] load .dockerignore 0.0s => => transferring context: 0.0s ------ > [internal] load build definition from...
I am wondering why this docker build command fails: docker build -t inline_img -f ERROR [internal] load build definition from 11 0.0s => => transferring dockerfile: 55B 0.0s => CANCELED [internal] load .dockerignore 0.0s => => transferring context: 0.0s ------ > [internal] load build definition from 11: ------ failed to solve with frontend dockerfile.v0: failed to read dockerfile: error from sender: failed to xattr /dev/fd/11: operation not permitted is this a perms issue?
Alexander Mills (10734 rep)
Aug 31, 2023, 08:18 PM • Last activity: Aug 31, 2023, 10:40 PM
130 votes
6 answers
54755 views
Process substitution and pipe
I was wondering how to understand the following: > Piping the stdout of a command into the stdin of another is a powerful > technique. But, what if you need to pipe the stdout of multiple > commands? This is where process substitution comes in. In other words, can process substitution do whatever pi...
I was wondering how to understand the following: > Piping the stdout of a command into the stdin of another is a powerful > technique. But, what if you need to pipe the stdout of multiple > commands? This is where process substitution comes in. In other words, can process substitution do whatever pipe can do? What can process substitution do, but pipe cannot?
Tim (106420 rep)
Jul 21, 2011, 03:59 AM • Last activity: Jul 6, 2023, 09:31 AM
3 votes
1 answers
280 views
Misunderstanding the purpose of a process substitution
I guess I'm missing some understanding on the use cases of a process substitution. My intuition was that a process substitution, of the form `<(COMMANDS)` would execute `COMMANDS` and then feed the result of the program into whatever command it's a part of, so `command1 <(command2)` would evaluate `...
I guess I'm missing some understanding on the use cases of a process substitution. My intuition was that a process substitution, of the form <(COMMANDS) would execute COMMANDS and then feed the result of the program into whatever command it's a part of, so command1 <(command2) would evaluate command2 and pass the result as the first argument into command1. I thought the following would've worked: $ for i in <(cat list.txt); do echo $i; done where list.txt is a file containing a list of words (separated by newlines). When I run this, it simply outputs /dev/fd/63, which I can only assume is like a temporary pathname to the output of the subshell created in the process substitution? I thought the above would've worked, because it works fine when I write $ for i in cat list.txt; do echo $i; done I've never seen this
`
notation before, what does it mean exactly? And what understanding am I lacking about process substitutions?
user3002473 (193 rep)
Feb 9, 2018, 02:01 AM • Last activity: Jun 14, 2023, 04:09 PM
1 votes
1 answers
441 views
Temporary symlink in shell - "named process substitution" - rename a file without creating a copy/symlink on the disk?
Thunderbird doesn't open files that don't have .eml extension as email files, but instead starts to compose a new message and adds them as attachments. To use thunderbird in scripts I'm looking for a way that would allow you to "temporarily rename" a `file` as a `file.eml`, open it in thunderbird (p...
Thunderbird doesn't open files that don't have .eml extension as email files, but instead starts to compose a new message and adds them as attachments. To use thunderbird in scripts I'm looking for a way that would allow you to "temporarily rename" a file as a file.eml, open it in thunderbird (possibly edit, but at least read) and close it without saving anything to the disk - all in that process' memory. While this could be achieved by either copying it or creating a symlink to that file in a temporary location there are many elegant ways to use process substitution or here strings that instead create "temporary files" in RAM that live only for duration of the process. Is there anything that could create such "pseudo symlinks" in bash/zsh?
Daniel Krajnik (371 rep)
Jun 6, 2023, 09:17 PM • Last activity: Jun 6, 2023, 10:51 PM
Showing page 1 of 20 total questions