I like to use set -x in scripts to show what's going on, especially if the script is going to run in a CI/CD pipeline and I might need to debug some failure post-hoc. One annoyance with doing this is that if I want to echo some text to the user (e.g., a status message or "I'm starting to do $X") then that message gets output twice - once for the echo command itself being echoed, and then once as the output of that echo command. What's a good way to make this nicer? One solution is this:

set -x

... bunch of normal commands that get echoed

(
  # Temporarily don't echo, so we don't double-echo
  set +x
  echo "Here is my status message"
)

... rest of commands get echoed again

But the two problems with that are 1. That's a lot of machinery to write every time I want to tell the user something, and it's "non-obvious" enough that it probably requires the comment every time 2. It echoes the set +x too, which is undesirable. Is there another option that works well? Something like Make's feature of prepending an @ to suppress echoing would be great, but I've not been able to find such a feature in Bash.

Assume the following C snippet showing a piece of error handling:

#include 
#include 
…
int main(int argc, char argv*[]) {
  int retval=0;
  …
  if(3!=argc) { /* wrong number of arguments */
    retval |= EX_USAGE;
    const int fprintf_retval = fprintf(stderr, "Bad syntax.\n");
    if (0>=fprintf_retval) retval |= … Hmm …  ;
  }
  …
  return retval;
}

Which error constant from sysexits.h should be placed instead of “… Hmm …”? We can say EX_OSERR (a failure to print an error message shows that something is off, though not necessarily deep in the operating system), or EX_IOERR (it's an output error, though not necessarily into a file), or EX_SOFTWARE (fairly general, but the reason for a failure to print an error typically lies outside the C program, and the error might relate to the operating system). Based only on the comments in sysexits.h, neither of the three constants fits perfectly. So what's the convention? EDIT: In the scope of this question, we assume that the programmer does want to discern between certain kinds of errors via the exit code, and that more than one error can occur.

I'm on an Ubuntu system which has a touchscreen set up on display :0. I wanted to be able to control this touchscreen remotely from Windows, so I set up x11vnc. It works great. But here's the setup: I have two users (excluding root). One restricted user (basically no permissions, and no password) who logs in automatically to the touchscreen, and one fully configured user with a complex password, sudo permission, etc. I only really care about starting x11vnc when the restricted user logs in (touchscreen becomes active), so I put the command x11vnc -loop & at the end of the file /home/restricted/.profile. Works fine. Note, however, that it has to be a background process because there's other commands that this user has to run (hidden from the human user) later on. The problem I'm having is that x11vnc is very noisy with its output. Especially if you don't use a password (which I don't), and it's a loop so it just keeps on repeating. Usually this wouldn't be an issue, but not only is redirecting the output to /dev/null not working (explained below), but the output is appearing on the other user's terminal when I log in via ssh to it (which is really annoying because I use that account for maintenance and its impossible to do anything when your screen is being filled with output of some background process. Here's an example of the output I get: --- x11vnc loop: 1 --- --- x11vnc loop: waiting for: 1551 ############################################################### #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@# #@ @# #@ ** WARNING ** WARNING ** WARNING ** WARNING ** @# #@ @# #@ YOU ARE RUNNING X11VNC WITHOUT A PASSWORD!! @# #@ @# #@ This means anyone with network access to this computer @# #@ may be able to view and control your desktop. @# #@ @# #@ >>> If you did not mean to do this Press CTRL-C now!! , -OR- set and export your $DISPLAY environment variable to refer to the display of the desired X server. - Usually the display is simply ":0" (in fact x11vnc uses this if you forget to specify it), but in some multi-user situations it could be ":1", ":2", or even ":137". Ask your administrator or a guru if you are having difficulty determining what your X DISPLAY is. ** Next, you need to have sufficient permissions (Xauthority) to connect to the X DISPLAY. Here are some Tips: - Often, you just need to run x11vnc as the user logged into the X session. So make sure to be that user when you type x11vnc. - Being root is usually not enough because the incorrect MIT-MAGIC-COOKIE file may be accessed. The cookie file contains the secret key that allows x11vnc to connect to the desired X DISPLAY. - You can explicitly indicate which MIT-MAGIC-COOKIE file should be used by the -auth option, e.g.: x11vnc -auth /home/someuser/.Xauthority -display :0 x11vnc -auth /tmp/.gdmzndVlR -display :0 you must have read permission for the auth file. See also '-auth guess' and '-findauth' discussed below. ** If NO ONE is logged into an X session yet, but there is a greeter login program like "gdm", "kdm", "xdm", or "dtlogin" running, you will need to find and use the raw display manager MIT-MAGIC-COOKIE file. Some examples for various display managers: gdm: -auth /var/gdm/:0.Xauth -auth /var/lib/gdm/:0.Xauth kdm: -auth /var/lib/kdm/A:0-crWk72 -auth /var/run/xauth/A:0-crWk72 xdm: -auth /var/lib/xdm/authdir/authfiles/A:0-XQvaJk dtlogin: -auth /var/dt/A:0-UgaaXa Sometimes the command "ps wwwwaux | grep auth" can reveal the file location. Starting with x11vnc 0.9.9 you can have it try to guess by using: -auth guess (see also the x11vnc -findauth option.) Only root will have read permission for the file, and so x11vnc must be run as root (or copy it). The random characters in the filenames will of course change and the directory the cookie file resides in is system dependent. See also: http://www.karlrunge.com/x11vnc/faq.html --- x11vnc loop: sleeping 2000 ms --- --- x11vnc loop: 2 --- And that just repeats on and on. It's yelling at me about not being able to open the display, but it works. I'm guessing this is because the restricted user already ran this command and it connected fine, but now this user is trying to do so as well, so it can't connect. Why is this happening? There's clearly some hole in my understanding. Here are the different things I tried: --- **Redirecting output to /dev/null** In the same place, /home/restricted/.profile I tried replacing the existing command x11vnc -loop & with x11vnc -loop > /dev/null 2>&1 &. This had zero effect, output was still on other user's terminal and (obviously) didn't go away. --- **Running command with nohup** Again in /home/restricted/.profile, I tried replacing the existing command x11vnc -loop & with nohup x11vnc -loop &. Once again, this had no effect at all. Output was still there. Nohup didn't even create a nohup.out file. At least, I couldn't find it. For sake of completion, I also tried nohup x11vnc -loop > /dev/null 2>&1 & again, nothing changed. --- **Starting x11vnc as a daemon** Here, I started realizing that I was doing something wrong. I figured that maybe it was because /home/restricted/.profile was the wrong place to put this kind of thing. So I made a script in /etc/init.d to be run on boot and made the link with update-rc.d following the example of the skeleton file. Once again, x11vnc worked, but the output was on all my terminals. --- **Starting x11vnc in screen** My last attempt at fixing this thing was throwing the command in a screen session. Screen usually fixes problems like this for me... But not this time. In /home/restricted/.profile, I replaced x11vnc -loop & with screen -d -R x11vnc x11vnc -loop This time, x11vnc didn't work anymore (wasn't able to connect via Windows client), but the output was STILL on all the terminals. --- Again, clearly there's something I'm not understanding either about x11vnc or spawning a process on boot. As you might have been able to tell from the above, I've been at this troubleshooting for a while and can't find anything else on the web about this. So help here would be awesome. Thanks in advance.

I have a bit of a problem making this script work and was wondering if anyone knew what the problem is. It is supposed to take a list of urls as input, scan them for sub directories and then output it to different files. for i in $(urls.txt); do echo $i; fn=$(echo $i | sed "s/\///g"); echo $fn; dirb $i /usr/share/wordlists/dirb/small.txt > $fn; done This script does correctly remove slashes for the new file but I get the error (translated from Swedish) that ": no such file or directory". Probably from the > $fn part. for i in $(urls.txt); do echo $i; fn=$(echo $i | sed "s/\///g"); echo $fn; dirb $i /usr/share/wordlists/dirb/small.txt -o $fn; done And trying to use dirb's own output I instead get an error like "OUTPUT_FILE httpwww.website.com (!) FATAL: Error opening output file: httpwww.website.com" And some clarification regarding the one liners: 1) loop through a list of urls 2) output the url to terminal 3) generate filename from url using sed 4) output filename to terminal 5) run url through dirb and try to output it to file name The supposed output file names look fine to me but aren't working. Does anyone know why that is?

My laptop (Kubuntu 18.04) has only a single sound card. I want to take input from the Mic of the headphone I connect but get the output to the built-in speaker of the laptop. I can toggle input (between internal and headphone mic) when headphone is connected, but if try to toggle output to speaker in the Pulseaudio Volume Control settings it shows "Unavailable" and if I switch to it, no sound comes out (i.e. the speaker works iff headphone is disconnected). I tried with Simultaneous Output but no luck.

I have an expect script which provides the IP address: #!/bin/expect -f set nodename [lindex $argv 0] spawn virsh console $nodename expect "Escape character is" send "\n" expect "localhost login: " { send "root\n" expect "Password: " send "cloud123\n" } expect "~]#" { send "\n" send "ifconfig | grep 192.168.1. | awk \'{print \$2}\'" send "\n" expect '(\d+\.\d+\.\d+\.\d+)' send "logout" } I want the script to return this IP address. I am calling this expect script from a shell script as below #!/bin/bash ip=$(expect GetIP.exp nodetwo) echo $ip How can I make my expect script return output to the shell script?

I'd like to figure out if an external program ran successfully and wrote something to stdout, or if it did not write anything, which happens in case an error occurred. The program unfortunately always returns with exit status 0 and no stderr output. The check should also not modify or hide whatever is being written to stdout by the program. --- If program output is short, it's possible to use command substitution to capture it, store it in an environment variable and print it again:

output="$(external_program)"
printf %s "$output"
if [ -z "$output" ]; then
    # error: external program didn't write anything
fi

Possible dealbreakers: - Depends on size of output, see https://unix.stackexchange.com/questions/357843/setting-a-long-environment-variable-breaks-a-lot-of-commands - Possibly modifies output, notably removes newlines, see https://unix.stackexchange.com/questions/164508/why-do-newline-characters-get-lost-when-using-command-substitution - If external program output is binary data, it's not a good idea to store that in an environment variable (?) --- Another possibility would be to write the output to a temporary file or pipe. This could be wrapped into a function which communicates the result via exit status:

output_exist() (
    temporary_file="$(mktemp)" || exit
    trap 'rm "$temporary_file"' EXIT
    tee "$temporary_file"
    test -s "$temporary_file"
)

if ! external_program | output_exist; then
    # error: external program didn't write anything
fi

Possible dealbreakers: - Temporary file or pipe that has to be taken care of, ensure clean-up etc., convenience vs. portability, e.g. POSIX only has mktemp(3), no mktemp(1) - Very large output may lead to resource/performance issues --- What alternatives are there? Is there a more straightforward solution?

I want to test if the output of a function matches the text I envision it should write to standard output. But the code below never prints

PASS

and I can't figure out why. I also tried putting various

\n

characters or deleting them in the variable

but that didn't help.

error_out()
{
    printf "%s\n" "${ERROR}"
    printf "Return to exit\n"
    read -r THROW_AWAY
    exit
}

ERROR="Test output."
OUTPUT_TO_TEST=$(error_out)
GROUND_TRUTH="Test output.\nReturn to exit\n"
{ [ "${OUTPUT_TO_TEST}" = "${GROUND_TRUTH}" ] \
         && printf "error_out test PASS\n"; } \
    || { \
        printf "error_out test FAIL\n"; \
    }

So I have 2 different Linux installations. One of them is Ubuntu and the second one is Kali. When I run

command with no options/arguments on my Ubuntu install I get:

michal@ubuntu:~$ date
Thu 24 Mar 2022 07:56:23 PM CET

When I run

command with no options/arguments on my Kali install I get:

┌──(michal㉿kali)-[~]
└─$ date
Thu Mar 24 07:58:34 PM CET 2022

The locale setting is the same on both machines being: Ubuntu

settings:

michal@ubuntu:~$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

and Kali

settings:

┌──(michal㉿kali)-[~]
└─$ locale
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Why the

command output is different on both machines? I want to PERMANENTLY change the Kali output, to be the same as my current Ubuntu output being:

michal@ubuntu:~$ date
Thu 24 Mar 2022 07:56:23 PM CET

Which file needs to be edited? Where are those settings? I've tried to follow steps from this thread: How can I change the default date format (using LC_TIME)? but I don't understand what: "

's texinfo also explicitly recommends to **set LC_TIME to C** in order to produce locale independent output." means.

I know that GNU Parallel buffers std/stderr because it doesn't want jobs output to be mangled, but if I run my jobs with parallel do_something ::: task_1 task_2 task_3, is there anyway for task_1's output to be displayed immediately, then after task_1 finishes, task_2's up to its current output, etc. If Parallel cannot solve this problem, is there any other similar program that could?

Let's say I have serverA with a block device, the contents of which I want to move to serverB's block device via the computer I am working on as a relay. This is very handy with pipes, buffers, dd and similar tools. In order to monitor the progress and identify any bottlenecks, I like to follow the progress of the individual status output of each command. Aside from complicating this one-liner by running multiple terminals, backgrounding (&), looping the outputs through FIFOs or the like, is there any way to handily sort the status outputs? ssh serverA "sudo dd if=/dev/sdc status=progress | mbuffer" | mbuffer | ssh serverB "mbuffer | sudo dd of=/dev/sdc status=progress" This gives me a single row that rapidly switches between the five different commands' status and also does not clean the row in between. For example alternating between the following (note the '00% full' on the alternate output): in @ 6137 kiB/s, out @ 6137 kiB/s, 26.3 GiB total, buffer 100% full 30353129472 bytes (30 GB, 28 GiB) copied, 1158 s, 26.2 MB/s00% full A neat output would keep each command's output consistently on a separate row.

I have a txt file with lines like below. Starting With: Parameters: {"raw_message"=>"MSH....... Ending With: "} I'm wanting to grep and output to a file the words between raw and transformed which appears halfway through each line. The file appears like below Parameters: {"raw_message"=>"MSH....... "transformed_data"....."} Parameters: {"raw_message"=>"MSH....... "transformed_data"....."} Parameters: {"raw_message"=>"MSH....... "transformed_data"....."} Parameters: {"raw_message"=>"MSH....... "transformed_data"....."} Best case scenario The MSH following the >MSH begins the output but there are many MSH instances in each line. So i figured it could be logical to grep the between message and then rip the raw and transformed part out. raw_message"=>"MSH......preceding words followed by transformed Some possible words preceding transformed LAB\r", "transformed 00355", "transformed So I'd want MSH....LAB\r MSH....00355 Any assistance would be much appreciated! I tried: sed -n "//,//p" HL7prod.txt > HL7prod2.txt Example Line Parameters: {"raw_message"=>"MSH|^~\\&||CDFGTL|||20144543000||ATG^A05|TLGTADM.1.13773085|P|2.1\rEVN|A08|11111111111|||MDFGQ8833^HLPS^GEGES^^^^\rPID|1||K11111111|K1111111|HOLVBVFS^LGDSA^^^^||19GHYSSD|F|^^^^^||^^^^^^^^|||||||K01045435547691\rPV1|1|P|K.ER^^||||LKIJK^Liujn^Jeggrs^H^^^MD|||ER||||||N||ER|||||||||||||||||||||DFLHL|ABD DFIN|PRE|||111111111||||||||\rZCS||^^^^||||00355", "transformed_data"=>"|^~\\&CDFLHCOCTL201506331000ADTA08TLGGBGM.1.13773076P2.1A08201506125500MDHYQ6633LUJKLJYTEDFG1K0567432372K5894336HOLDFGEERAAAAS1111111111FK01013333333331PK.ERJTOLOKSLdasfsJtuygikdHMDERNERCOTOLAABD XXXXPRE111111111111111COTYULCDFTLK.ER"} Would Want: MSH|^~\\&||CDFGTL|||20144543000||ATG^A05|TLGTADM.1.13773085|P|2.1\rEVN|A08|11111111111|||MDFGQ8833^HLPS^GEGES^^^^\rPID|1||K11111111|K1111111|HOLVBVFS^LGDSA^^^^||19GHYSSD|F|^^^^^||^^^^^^^^|||||||K01045435547691\rPV1|1|P|K.ER^^||||LKIJK^Liujn^Jeggrs^H^^^MD|||ER||||||N||ER|||||||||||||||||||||DFLHL|ABD DFIN|PRE|||25679506645657||||||||\rZCS||^^^^||||00355

I have a command which outputs lots of data (say, strace with lots of syscalls, running for a few minutes). Is there any option (e.g. command wrapper or something similar) that would allow me to pause the output of the command (just the output on the screen, I don't mind the command running in the background), then unpause it after I take a look on its output?

I have a find command that I exec to shred and it works great; however, I need to capture the output of this and pass to a variable or a file. I have tried so many iterations of the below commands, but cant seem to get it to work. I also have tried for loops. It always will display to my terminal as it goes through the shred process, but never writes to a file or variable. I would prefer a variable but will take anything at this point. Please see my examples below: **Original find command:** find /burncd/working -type f -exec shred -v -n7 -z --remove {} \; **Latest try at a command to pass to a file (from StackExchange):** find /burncd/working/* -type f -exec bash -c 'shred -v -n7 -z --remove "$1" /tmp/find.out' $0 {} \; I would appreciate any help you could give.

I am on elementaryOS 0.4.1 Loki, which is based on Ubuntu 16.04.1. I want to use OBS , a screen recorder, to record gameplay along with the sounds that come from the same video game. I also want to use a voice chat application in the background while playing the game, but I do not want any of it recorded by OBS. OBS can't selectively ignore the audio of certain programs, but it can record sound from a specified input device. **I want to create a virtual input and use pavucontrol's output list to route my voice chat application to that input so it's not picked up by OBS. At the same time, I want this input to play back to a headphone/line out port.** I know this can be done in Windows with software such as Virtual Audio Cables, but I don't know how to do the same thing in Linux. I already attempted to do something with sudo modprobe snd-dummy but it doesn't let me route the dummy to a headphone output. How should I approach this?

I am trying to use part of the hostname output in Linux and use it in a file using sed. For example, I am using this command as hostname | tail -c 4 which shows me output as last 4 numbers and then use this output and substitute it with another text inside a file. Assuming part of the hostname command shows with tail -c 4 option, "1234" How can I take it this further and use it to replace it with another file? I can do this manually by: hostname | tail -c 4 ; sed -i 's/oldtext/1234/g' filename.txt but not sure how can I achieve this with script? Any ideas?

Hy, Is it possible to customize nftables log's to add more informations ? For example, it could be interesting to get users ( id, gid, ... ) or processes informations that try to out. Example of classic log : IN= OUT=ens18 SRC=192.168.1.2 DST=8.8.8.8 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=64026 DF PROTO=TCP SPT=51096 DPT=8888 WINDOW=64240 RES=0x00 SYN URGP=0 Regards

I have 2 exactly same formatted, same size and same brand SD-cards. I would like to dd image to /dev/disk2 and to /dev/disk3 at the same time. Pseudocode sudo dd bs=1m if=/Users/masi/2016-05-10-raspbian-jessie.img of={/dev/disk2,/dev/disk3} --- How can you dd from one input to many output SDs?

I write a lot of non-interactive scripts where I would like all output to go to a log file and have nothing appear on-screen. To solve this, I've been using: #!/bin/bash exec &> logfile echo "Step One:" /some/command/one echo "Step Two:" /some/command/two I want to make sure this is a sane method. Are there any significant drawbacks or issues I'm going to encounter if I move forward with this methodology? If so, what are they and how best can they be mitigated (including by changing my methodology).

Scenario:

$ cat libs.txt
lib.a
lib1.a

$ cat t1a.sh
f1()
{
        local lib=$1
        stdbuf -o0 printf "job for $lib started\n"
        sleep 2
        stdbuf -o0 printf "job for $lib done\n"
}
export -f f1
cat libs.txt | SHELL=$(type -p bash) parallel --jobs 2 f1

Invocation and output:

$ time bash t1a.sh
job for lib.a started
job for lib.a done
job for lib1.a started
job for lib1.a done

real    0m2.129s
user    0m0.117s
sys     0m0.033s

Here we see that execution of f1 was indeed in parallel (real 0m2.129s). However, diagnostic output looks like execution was sequential. I expected the following diagnostic output:

job for lib.a started
job for lib1.a started
job for lib.a done
job for lib1.a done

Why does diagnostic output look like sequential execution rather than parallel execution? How to fix the diagnostic output so that it does look like parallel execution?