Take the following Bash script 3-output-writer.sh:

lang-bash
echo A >&1
echo B >&2
echo C >&3

Of course when ran as . 3-output-writer.sh it gets the error 3: Bad file descriptor, because Bash doesn't know what to do with the 3rd output stream. One can easily . 3-output-writer.sh 3>file.txt, though, and Bash is made happy. But here's the question: how do I pipe all these into another process, so that it would have all three to work with? Is there any way other than creating three named pipes, as in,

lang-bash
mkfifo pipe1 pipe2 pipe3  # prepare pipes
. 3-output-writer.sh 1>pipe1 2>pipe2 3>pipe3 &  # background the writer, awaiting readers
3-input-reader pipe1 pipe2 pipe3  # some sort of reader, careful not to block
rm pipe1 pipe2 pipe3

?

The so-called "standard streams" in Linux are stdin, stdout, and stderr. They must be called "standard" for a reason. Are there non-standard streams? Are those non-standard streams fundamentally treated differently by the kernel?

I have a number of video files (500+) with lots of audio and subtitle streams for languages that I don't need and would thus like to remove to conserve storage space. I tinkered around with ffmpeg, but removing streams by processing one file after another turned out to *very* time consuming. Had no luck with scripting either as the video files contain different streams in different orders, which makes removal via index difficult and error prone. **There must be a solution that is both faster and also works for files containing different streams, right?** Any help would be much appreciated.

I'm trying to concatenate multiple input files/streams into one stream (using the imaginary command stream-cat), pipe that stream into ssh and on the remote host separate it back into individual files/streams (stream-sep), like in this example, which is for demonstration purposes only:

stream-cat ( zfs receive tank/vm@snapshot ) somefile.txt"

Explanation of the example: zfs send outputs a large stream of data whose size isn't known in advance (that's why tar can't handle it). That data stream is concatenated with the content of a regular file somefile.txt. The resulting stream is piped into ssh, where it is separated again. The first stream gets piped into zfs receive, while the second is written to a regular file. Such a program should be straightforward to implement by reading non-seekable streams in chunks and always writing the chunk size followed by the data, until the end of the stream is reached. The overhead would be minimal. Does such a program exist already?

It's well known that redirecting standard output and error to the same file with cmd >out_err.txt 2>out_err.txt can lead to loss of data, as per the example below:

work:/tmp$ touch file.txt
work:/tmp$ ls another_file.txt
ls: cannot access 'another_file.txt': No such file or directory

The above is the setup code for the example. An empty file file.txt exists and another_file.txt is not a thing. In the code below, I naively redirect to out_err.txt both input and output os listing these files.

work:/tmp$ ls file.txt another_file.txt >out_err.txt 2>out_err.txt
work:/tmp$ cat out_err.txt 
file.txt
t access 'another_file.txt': No such file or directory

And we see that we lost a few characters in the error stream. However, using >> works in the sense that replicating the example would yield keep the whole output and the whole error. Why and how does cmd >>out_err.txt 2>>out_err.txt work?

I have a device under test (DUT) and I measure its power usage the using a Power Analyzer Datalogger using the data from /dev/ttyUSB0. The problem is that the DUT is now remotely from the workstation I used to gather data with, but in the same network, I need to use a 2nd PC which is directly connected via USB to the Power Analyzer as a sort of USB proxy and ssh to create a kind of symbolic link on the measuring machine of the USB of the "proxy" machine. Given the above diagram how can the 1st PC access /dev/ttyUSB0 of the 2nd PC which is directly connected, in a way that a program reading the stream from the 1st PC will not notice the difference?

I'm trying to spawn a new terminal, execute a few commands and pipe their output to stdin of the original process. A mwe of what I'm trying to do is the following bash one-liner:

$ xterm -e sh -c "echo -e 'foo\nbar' > /proc/$$/fd/1" | grep foo

where I spawn a new xterm window, print foo\nbar and try to redirect it to stdin of the shell I'm executing this command from. To see if it works I then pipe it into grep. The code above prints both foo and bar to stdout instead of just foo, but I'm not sure why. How can I fix this? EDIT: the real case implementation of this is a file picker that uses fzf:

#!/usr/bin/env bash

function open() {
  # takes a bunch of file names (either passed as arguments or from stdin) and
  # opens them
  true
}

# This is what I have now
alacritty \
   -e sh -c 'fzf -m --prompt="Open> " --border=horizontal --print0 \
              | (nohup xargs -0 bash -c '\''open "$@"'\'' _ &>/dev/null &)'

# This is what I'm trying to do
# alacritty \
#  -e sh -c "fzf -m --prompt='Open> ' --border=horizontal > /proc/$$/fd/1" \
#  | open

Tried ping between computer's static IP's connected by router (General failure). Therefore UDP direct connection attempts went nowhere. Tried using ffplay and pulse to stream audio, but Debian 8 is finicky with PulseAudio. AlsaMixer displays that there is an analog audio device, and the sound works, and the device is Pulse. Most pactl arguments return "Connection refused, Access Denied." Tried setting default sink in system.pa and default.pa to HDMI source but the `pactl list sinks` shows HDMI is Suspended. It turns out I have a capture card in the system I want to get audio to, but the HDMI is "unplugged" in pavucontrol. An AMD and NVIDIA GPU were tried. If I were to get networking going between the two computers, since the configuration is wired LAN, going back to streaming audio as the solution would be effective. That or perhaps getting the HDMI capture card to capture audio, as it does capture video. Rather, a different Linux distro entirely.

I know that the stream for file descriptor 2u is stderr. However, I'm not sure what the stream for file descriptor 3u would be and why? Ultimately, what does 3u represent here?

There is an example that streams a webcam to another computer by piping a V4L2 stream into netcat (v4l2-ctl --stream-to - | nc ), and this works, but it feels redundant when v4l2-ctl --stream-to-host [:] exists. I was able to capture and display the --stream-to-host stream via qvidcap -p as described in the v4l2-ctl man page example section , but am stumped when trying to pipe it into v4l2-loopback (via FFmpeg) for use with video conferencing software on the receiving machine. I have tried various combinations of parameters to v4l2-ctl, including --stream-from-host, --stream-loop, --stream-poll and --stream-mmap, but v4l2-ctl --help-all is too terse for me. Ironically, I was able to receive a stream (albeit garbled beyond recognition) when receiving with netcat and piping it to ffmpeg (nc -l | ffmpeg -f v4l2 /dev/video0. How can I make an incoming stream created via v4l2-ctl --stream-mmap --stream-to-host [:] available to video conferencing software via v4l2-loopback?

Conceptually, a stream is a sequence of "characters" or "atoms", i.e. a binary stream is a sequence of 0s and 1s. But in Linux standard streams, if I write a bash script that asks "read", then I think it treats a single line (ending with "ENTER") as a "character", but I'm not sure. This suggests to me that a single "atom" is a string, and that atoms are delimited by ENTER. Also, I assume that for other programs, they don't take strings as input, but other data-types. Am I on the right track? what are the atoms/characters in a standard stream and how does the program know how to carve up a file into atoms?

Hopefully this question isn't too abstract, it touches on a bunch of software throughout the stack. We all know about /dev/stdin, /dev/stdout, and /dev/stderr. What if I wanted to create a new standard stream, /dev/stdjson? What software would need to support that? I'm assuming that I don't *need* to pass a file descriptor to /dev/stdjson to every program, that they can open that stream on an as-needed basis. So where could you add a new standard stream? Would it need to be a kernal module, a part of the shell?

Let's say that there is one huge **db.sql.gz** of size 100GB available https://example.com/db/backups/db.sql.gz and the server supports range requests . So instead of downloading the entire file, I downloaded **y** bytes(let's say 1024bytes) with an offset of **x** bytes(let's say 1000bytes) like the following. curl -r 1000-2024 https://example.com/db/backups/db.sql.gz With the above command I was able to download the partial content of the gzipped file, now my question is how can I read that partial content? I tried gunzip -c db.sql.gz | dd ibs=1024 skip=0 count=1 > o.sql but this gives an error >gzip: dbrange.sql.gz: not in gzip format The error is acceptable since I guess at the top of the file may be there are header blocks which describes encoding. ------- I noticed that if I'm downloading the file without an offset, I'm able to read the file using gunzip and piping. curl -r 0-2024 https://example.com/db/backups/db.sql.gz

I have a script that loops over some big collection of data and performs some lenghty operations. Then i need to sort | uniq -c its output. So to let it know that its alive, I print a dot every N items on stderr (very primitive pseudo progress-bar), so it looks pretty much like this:

for i in {1..100}; do 
    [[ $(( (i+=1) % 10)) -eq 0 ]] && echo -n "." >&2
    shuf -i 1-10 -n1
    sleep 0.1
done | sort | uniq -c

and the output:

..........      9 1
     10 10
      8 2
     14 3
     13 4
      9 5
     11 6
      8 7
      8 8
     10 9

the "progress bar" messes up the output a little - so i was wondering: * is there an easy way to add a nweline to that stderr before flushing that stdout? (probably echo >&2 is all I need) * or remove it ? of course in reeality i dont know wow many items there are (at least not out-of-the box). So i was wondering if this can be acieved by some stream redirection

I have dd from GNU coreutils 8.32. When I run { echo a; sleep 1; echo b; } | dd bs=4 count=1 then I get a 0+1 records in 0+1 records out 2 bytes copied, 2.0381e-05 s, 98.1 kB/s dd terminates during the sleep even though the block size was not reached and there was no EOF. The output b\n is lost. This does not happen if I remove either sleep or count=1. In man dd I couldn't find anything that describes this behavior. 1. Why doesn't dd count=1 wait till bs is reached or an EOF is encountered? 2. How can I force dd to wait?

What explains the following inconsistency in the reported contents of /dev/fd?

erhannis@mnode6:/dev/fd$ ll /dev/fd/
total 0
dr-x------ 2 erhannis erhannis  0 Jan 12 22:10 .
dr-xr-xr-x 9 erhannis erhannis  0 Jan 12 22:10 ..
lrwx------ 1 erhannis erhannis 64 Jan 12 22:10 0 -> /dev/pts/8
lrwx------ 1 erhannis erhannis 64 Jan 12 22:10 1 -> /dev/pts/8
lrwx------ 1 erhannis erhannis 64 Jan 12 22:10 2 -> /dev/pts/8
lr-x------ 1 erhannis erhannis 64 Jan 12 22:10 3 -> /proc/24334/fd
erhannis@mnode6:/dev/fd$ ll
total 0
dr-x------ 2 erhannis erhannis  0 Jan 12 21:42 .
dr-xr-xr-x 9 erhannis erhannis  0 Jan 12 21:42 ..
lrwx------ 1 erhannis erhannis 64 Jan 12 21:42 0 -> /dev/pts/8
lrwx------ 1 erhannis erhannis 64 Jan 12 21:42 1 -> /dev/pts/8
lrwx------ 1 erhannis erhannis 64 Jan 12 21:42 2 -> /dev/pts/8
lrwx------ 1 erhannis erhannis 64 Jan 12 21:55 255 -> /dev/pts/8

Specifically, note that ll alone shows a file 255 linked to /dev/pts/8, but ll /dev/fd/ shows instead a file 3 linked to /proc/24334/fd. (The proc number changes each time; I suspect it to be the process number of ll itself.) ...I also note, now, that the dates are different - am I getting a different directory for . than for /dev/fd/? I further note that I can't create a file in /dev/fd (with either path).

I am kinda new to Linux. My base issue: I need to show the output of curl cheat.sh/python like a man or less page, so that I can easily scroll up and down. What have I tried: curl cheat.sh/python | less outout: some weird text like this- > ESC38;5;246m# Python is a high-level programming languageESC[39m > ESC[38;5;246m# and python is a Python interpreter.ESC[39m > > ESC[38;5;246m# Basic example of server with pythonESC[39m > ESC[38;5;246m# Will start a Web Server in the current directory on > port 8 000ESC[39m ESC[38;5;246m# go to http://127.0.0.1:8000ESC[39m > ESC[38;5;246m#ESC[39m ESC[38;5;246m# Python v2.7ESC[39m > ESC[38;5;252mpythonESC[39mESC[38;5;252m > ESC[39mESC[38;5;252m-mESC[39mESC[ 38;5;252m > ESC[39mESC[38;5;252mSimpleHTTPServerESC[39m ESC[38;5;246m# Python > 3ESC[39m ESC[38;5;252mpythonESC[39mESC[38;5;252m > ESC[39mESC[38;5;252m-mESC[39mESC[ 38;5;252m > ESC[39mESC[38;5;252mhttp.serverESC[39mESC[38;5;252m ESC[39mESC[ > 38;5;67m8000ESC[39m ... After having a look at [this , I tried: curl -vs cheat.sh 2>&1 | less but output was kinda same: > Accept: */* > * Mark bundle as not supporting multiuse HTTP/1.1 200 OK Server: nginx/1.13.12 Date: Wed, 25 Mar 2020 17:50:04 GMT Content-Type: > text/plain; charset=utf-8 Content-Length: 25502 Connection: keep-alive > Strict-Transport-Security: max-age=63072000; includeSubdomains > X-Frame-Options: DENY X-Content-Type-Options: nosniff { [2429 bytes data] > ESC[38;2;0;204;0m_ESC[0mESC[38;2;0;204;0m_ESC[0mESC[38;2;0;204;0m > ESC[0mESC[38;2;0;204;0m ESC[0mESC[38;2;0; 204;0m ESC[0m > ___| |__ ___ __ _| |_ ___| |__ ESC[38;2;0;204;0m\ESC[0mESC[38;2;0;204;0m > ESC[0mESC[38;2;0;204;0m\ESC[0mESC[38;2;0;204;0m ESC[0mESC[38;2;0; > 204;0m ESC[0m ESC[48;2;85;85;85m ESC[0mESC[48;2;85;85;85m > ESC[0mESC[48;2;85;85;85m > ESC[0mESC[48;2;85;85;85mTESC[0mESC[48;2;85;85;85mhESC[0mESC[48 > ;2;85;85;85meESC[0mESC[48;2;85;85;85m > ESC[0mESC[48;2;85;85;85moESC[0mESC[48;2;85;85;85mnESC[0mESC[48;2;85;85;85mlESC[0mESC[48;2;85;85;85myESC[0m > ESC[48;2;85;85;85m > ESC[0mESC[48;2;85;85;85mcESC[0mESC[48;2;85;85;85mhESC[0mESC[48;2;85;85;85meESC[0mESC[48;2;85;85;85maESC[0mESC[48;2;85;85;85mt > ESC[0mESC[48;2;85;85;85m > ESC[0mESC[48;2;85;85;85msESC[0mESC[48;2;85;85;85mhESC[0mESC[48;2;85;85;85meESC[0mESC[48;2;85;85;85meESC[0mESC[48;2;85;85 > ;85mtESC[0mESC[48;2;85;85;85m > ESC[0mESC[48;2;85;85;85myESC[0mESC[48;2;85;85;85moESC[0mESC[48;2;85;85;85muESC[0mESC[48;2;85;85;85m > ESC[0mESC[48;2; > 85;85;85mnESC[0mESC[48;2;85;85;85meESC[0mESC[48;2;85;85;85meESC[0mESC[48;2;85;85;85mdESC[0mESC[48;2;85;85;85m > ESC[0mESC[48;2;85;85;85m ESC[0mESC[ 48;2;85;85;85m ESC[0m ... Finally if I change the above code like (this is probably the wrong approach) : curl -vs cheat.sh 1>&2 | less Then I will get the needed output of curl, but upon pressing a button the terminal shows like: > ~ > ~ > ~ > ~ > ~ > ~ > ~ > ~ > ~ > ~ > ~ (END) and I can basically press q to quit out of less from there.

I know thats a pretty dumb question but I didn't found this precise question on internet I try to tar -cvjf all the contents of a directory (/*) and directly redirect that to a file (> file) but the title error message occurs. I compress both files and directories here

I had been doing some work with an old SVR4 box with serial i/o when I found that the driver for the serial card did not support TIOCMGET through an ioctl call (e.g. ioctl(fd, TIOCMGET, &arg);). Having the source for the driver, and looking at it - it didn't seem too hard to add an answer to the call. However, I've run into a bit of a snag in that what I am doing doesn't seem to work. Looking at the driver, it has the following code to service TCGETS: case TCGETS: { /* immediate parm retrieve */ register struct termios *cb; if (mp->b_cont) /* Bad user supplied parameter */ freemsg(mp->b_cont); if (!(bp1 = allocb(sizeof(struct termios), BPRI_MED))) { putbq(q, mp); bufcall(sizeof(struct termios), BPRI_MED, getoblk, (long)tp); return; } mp->b_cont = bp1; cb = (struct termios *)mp->b_cont->b_rptr; cb->c_iflag = tp->t_iflag; cb->c_oflag = tp->t_oflag; cb->c_cflag = tp->t_cflag; mp->b_cont->b_wptr += sizeof(struct termios); mp->b_datap->db_type = M_IOCACK; iocbp->ioc_count = sizeof(struct termios); putnext(RD(q), mp); break; } My thought was simply to copy this code and rather than return a termios structure, just return the int. As such, my code is similar: case TIOCMGET: { /* immediate parm retrieve */ register int *cb; if (mp->b_cont) /* Bad user supplied parameter */ freemsg(mp->b_cont); if (!(bp1 = allocb(sizeof(int), BPRI_MED))) { putbq(q, mp); bufcall(sizeof(int), BPRI_MED, getoblk, (long)tp); return; } mp->b_cont = bp1; cb = (int *)mp->b_cont->b_rptr; /* my original attempt to get some bits */ *cb = (ql->carrier * TIOCM_CAR | ql->rts * TIOCM_RTS | (ql->lp->command & 1) * TIOCM_DTR) /* Tried this to debug: *cb = 0; Doesn't seem to change the variable I pass in */ /* Tried this, compiles fine, *(int *)mp->b_cont->b_rptr = 0; but I get an improper argument passed error while running */ /* qreply(q, mp); not in my code, was a note to try it */ mp->b_cont->b_wptr += sizeof(int); mp->b_datap->db_type = M_IOCACK; iocbp->ioc_count = sizeof(int); putnext(RD(q), mp); break; } Making the call as shown previously, ioctl(fd, TIOCMGET, &arg), the arg value does not seem to get changed. I've tried a couple different attempts to return a value of 0 in case the problem was in my bit assignment code. However, I've not had any luck. I did write a program to make sure that TCGETS works as it should - and it does. So I'm not sure where I'm going wrong - probably something incredibly stupid and right in front of me. Hopefully that it is SVR4 and STREAMS isn't so arcane to put an answer out of reach. Thanks to all who look and try to help. Mack

I'ld like to hide ugly data from being shown by command line tools like cat (and maybe simple text editors too) which often get confused by binary data. For example a VT100 terminal sometimes gets misconfigured by binary outputs. cat or similar do not get trash on their screen.