time is a brilliant command if you want to figure out how much CPU time a given command takes. I am looking for something similar that can list the files being accessed by a program and its children. Either in real time or as a report afterwards. Currently I use: #!/bin/bash strace -ff -e trace=file "$@" 2>&1 | perl -ne 's/^[^"]+"(([^\\"]|\\[\\"nt])*)".*/$1/ && print' but its fails if the command to run involves sudo. It is not very intelligent (it would be nice if it could only list files existing or that had permission problems or group them into files that are read and files that are written). Also strace is slow, so it would be good with a faster choice.

I own two systems, host-a and host-b, connected in a lan via a switch; they both have no firewall. When I run clockdiff in each of them I receive the error message that the other host is down. All commands are executed in a clean bash by running exec -c bash --norc (files redacted).

root@host-a:~# clockdiff host-b
...........................clockdiff: host-b is down
root@host-a:~# clockdiff -o host-b
...........................clockdiff: host-b is down

At the same time, a tcpdump on the same machine host-a reveals that the queried host responds with ICMP messages (ICMP time stamp in the former case, ECHO in the latter)

root@host-a:~# tcpdump icmp
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on lan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
 ....
:18.078956 IP host-b > host-a: ICMP time stamp reply id 29228 seq 7424: org :18.078, recv :08.347, xmit :08.347, length 20
:18.078956 IP host-b > host-a: ICMP time stamp reply id 29228 seq 7680: org :18.078, recv :08.347, xmit :08.347, length 20
1
 ...

Curiously, when I run the exact same commands under strace, the replies are measured and a correct response is returned.

root@host-a:~# strace -o /tmp/log clockdiff host-b
....................................................
host=host-b rtt=0(0)ms/0ms delta=-9737ms/-9737ms 
root@host-a:~# strace -o /tmp/log clockdiff -o host-b
...................................................
root@host-a:~# strace -o /tmp/log clockdiff  host-b
....................................................
host=host-b rtt=0(0)ms/0ms delta=-9737ms/-9737ms 
root@host-a:~# uname -a
Linux host-a 6.1.0-25-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.106-3 (2024-08-26) x86_64 GNU/Linux

Some excerpt from the strace log:

...
connect(3, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 0
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=0}, NULL, 8) = 0 (Timeout)
sendto(3, "\r\0"..., 20, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 20
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=1, tv_nsec=0}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=999837227})
recvfrom(3, "E\0"..., 1024, 0, NULL, 0x7ffff369ecd8) = 40
newfstatat(1, "", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0xb), ...}, AT_EMPTY_PATH) = 0
write(1, ".", 1)                        = 1
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=937000000}, NULL, 8) = 0 (Timeout)
sendto(3, "\r\0"..., 20, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 20
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=937000000}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=936829484})
recvfrom(3, "E\0"..., 1024, 0, NULL, 0x7ffff369ecd8) = 40
write(1, ".", 1)                        = 1
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=842000000}, NULL, 8) = 0 (Timeout)
sendto(3, "\r\0"..., 20, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 20
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=842000000}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=841996498})
recvfrom(3, "E\0"..., 1024, 0, NULL, 0x7ffff369ecd8) = 40
write(1, ".", 1)                        = 1
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=736000000}, NULL, 8) = 0 (Timeout)
sendto(3, "\r\0"..., 20, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 20
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=736000000}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=735845402})
recvfrom(3, "E\0"..., 1024, 0, NULL, 0x7ffff369ecd8) = 40
write(1, ".", 1)                        = 1
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=631000000}, NULL, 8) = 0 (Timeout)
sendto(3, "\r\0"..., 20, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 20
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=631000000}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=630791917})
recvfrom(3, "E\0"..., 1024, 0, NULL, 0x7ffff369ecd8) = 40
write(1, ".", 1)                        = 1
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=532000000}, NULL, 8) = 0 (Timeout)
sendto(3, "\r\0"..., 20, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 20
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=532000000}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=531768694})
recvfrom(3, "E\0"..., 1024, 0, NULL, 0x7ffff369ecd8) = 40
write(1, ".", 1)                        = 1
sendto(3, "\r\0"..., 20, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("")}, 16) = 20
ppoll([{fd=3, events=POLLIN|POLLHUP}], 1, {tv_sec=0, tv_nsec=442000000}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=441949198})
recvfrom(3, "E\0"..., 1024, 0, NULL, 0x7ffff369ecd8) = 40
write(1, ".", 1)                        = 1
 ...

The excutable is not setuid and the OS is debian.

root@host-a:~# ls -laF which clockdiff
-rwxr-xr-x 1 root root 23120 Sep 24  2024 /usr/bin/clockdiff*
root@host-a# getcap /usr/bin/clockdiff                                
/usr/bin/clockdiff cap_net_raw=ep
root@host-a:~# dpkg -S /usr/bin/clockdiff
iputils-clockdiff: /usr/bin/clockdiff
root@host-a:~# clockdiff -V
clockdiff from iputils 20221126
libcap: yes, IDN: yes, NLS: no, error.h: yes, getrandom(): yes, __fpending(): yes

I have encountered strace Heisenbugs before, but in this case, I cannot fathom why the normal invocation would fail to see any responses, when it is still run as root. Also, this behaviour is empirically always reproducible in around 10 repetitions of the test. Why on earth can this be?

On one of my ubuntu server, top command start very slow, when I run top command on terminal, it will display info after more than 10 seconds. Then I use strace -yy top to analysis the problem, I found top hang at the following point:

getsockopt(6, SOL_SOCKET, SO_SNDBUF, , ) = 0                                                                 [0/1911]
setsockopt(6, SOL_SOCKET, SO_SNDBUFFORCE, , 4) = 0
connect(6, {sa_family=AF_UNIX, sun_path="/run/dbus/system_bus_socket"}, 29) = 0
getsockopt(6366381191]>, SOL_SOCKET, SO_PEERCRED, {pid=1, uid=0, gid=0}, ) = 0
getsockopt(6366381191]>, SOL_SOCKET, SO_PEERSEC, "unconfined", [64->10]) = 0
getsockopt(6366381191]>, SOL_SOCKET, SO_PEERGROUPS, "", [256->0]) = 0
fstat(6366381191]>, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
getsockopt(6366381191]>, SOL_SOCKET, SO_ACCEPTCONN, , ) = 0
getsockname(6366381191]>, {sa_family=AF_UNIX}, [128->2]) = 0
geteuid()                               = 0
sendmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0AUTH EXTERNAL ", iov_len=15}, {iov_base="30", iov_len
=2}, {iov_base="\r\nNEGOTIATE_UNIX_FD\r\nBEGIN\r\n", iov_len=28}], msg_iovlen=3, msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 45
gettid()                                = 25127
getrandom("\x00\x3c\x86\xd4\x34\xa0\xb2\x91\xee\x2d\x4b\x2c\x1b\x56\xda\xa8", 16, GRND_NONBLOCK) = 16
futex(0x7f00826d9ff0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
recvmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="OK 141ec8c80f29bf83dc576f5e61641"..., iov_len=256}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_CMSG_CLOEXEC) = 52
sendmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\1\0\1\0\0\0\0\1\0\0\0m\0\0\0\1\1o\0\25\0\0\0/org/fre"
..., iov_len=128}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 128
recvmsg(6366381191]>, {msg_namelen=0}, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable
)
ppoll([{fd=6366381191]>, events=POLLIN}], 1, {tv_sec=24, tv_nsec=999897000}, NULL, 8) = 1 ([{fd=6, revents=POLLIN}], left {tv_sec=24, tv_nsec=999818992})
recvmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\2\1\1\17\0\0\0\1\0\0\0E\0\0\0\6\1s\0\n\0\0\0", iov_le
n=24}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_CMSG_CLOEXEC) = 24
recvmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base=":1.2843938\0\0\0\0\0\0\5\1u\0\1\0\0\0\10\1g\0\1s\0\0"..
., iov_len=79}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_CMSG_CLOEXEC) = 79
sendmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\1\0\1\4\0\0\0\2\0\0\0\247\0\0\0\1\1o\0\31\0\0\0/org/f
re"..., iov_len=184}, {iov_base="w\364\0\0", iov_len=4}], msg_iovlen=2, msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 188
recvmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\4\1\1\17\0\0\0\2\0\0\0\225\0\0\0\1\1o\0\25\0\0\0", io
v_len=24}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_CMSG_CLOEXEC) = 24
recvmsg(6366381191]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="/org/freedesktop/DBus\0\0\0\2\1s\0\24\0\0\0"..., iov_le
n=159}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_CMSG_CLOEXEC) = 159
recvmsg(6366381191]>, {msg_namelen=0}, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable
)
ppoll([{fd=6366381191]>, events=POLLIN}], 1, {tv_sec=24, tv_nsec=999853000}, NULL, 8

And the command ls -l /proc/$(pgrep top)/fd/ get the following output:

lrwx------ 1 root root 64 1月  25 18:11 0 -> /dev/pts/9
lrwx------ 1 root root 64 1月  25 18:11 1 -> /dev/pts/9
l-wx------ 1 root root 64 1月  25 18:11 2 -> /dev/null
lrwx------ 1 root root 64 1月  25 18:11 3 -> /dev/pts/9
lr-x------ 1 root root 64 1月  25 18:11 4 -> /proc/uptime
lr-x------ 1 root root 64 1月  25 18:11 5 -> /proc
lrwx------ 1 root root 64 1月  25 18:11 6 -> 'socket:'

lsof -d 6 -U -a +E -p $(pgrep top) get following output:

COMMAND     PID       USER   FD   TYPE             DEVICE SIZE/OFF      NODE NAME
dbus-daem   874 messagebus   12u  unix 0xffff9545f6fee400      0t0 366381191 /var/run/dbus/system_bus_socket type=STREAM ->INO=366379599 25127,top,6u
top       25127       root    6u  unix 0xffff9545f6fefc00      0t0 366379599 type=STREAM ->INO=366381191 874,dbus-daem,12u

After top works as normal, the command ls -l /proc/$(pgrep top)/fd/ get the following output:

lrwx------ 1 root root 64 1月  25 18:11 0 -> /dev/pts/9
lrwx------ 1 root root 64 1月  25 18:11 1 -> /dev/pts/9
l-wx------ 1 root root 64 1月  25 18:11 2 -> /dev/null
lrwx------ 1 root root 64 1月  25 18:11 3 -> /dev/pts/9
lr-x------ 1 root root 64 1月  25 18:11 4 -> /proc/uptime
lr-x------ 1 root root 64 1月  25 18:11 5 -> /proc/meminfo
lr-x------ 1 root root 64 1月  25 18:11 6 -> /proc/loadavg
lr-x------ 1 root root 64 1月  25 18:12 7 -> /proc/stat

lsof -d 6 -U -a +E -p $(pgrep top) output nothing. I want to know the reason that why top command start so slow, can anyone help me?

So I am writing a program that automatically determines the dependencies of an application and writes a *FROM scratch* dockerfile based on them using *strace*. I was testing it on a MariaDB, but it failed because *chmod* was not found In the MariaDB GitHub page I can see that there is a [docker-entrypoint.sh](https://github.com/MariaDB/mariadb-docker/blob/master/main/docker-entrypoint.sh) which does a *find .. -exec chown mysql {} \;* but in the strace output i don't see an *execve("/bin/chown",...)* To trace the apps, I am using a *statically-linked strace* binary which I am mounting to the Docker container running the app alongside an out.log file which captures the output The full command is the following:

run --rm --entrypoint "" -v /usr/local/bin/strace:/usr/bin/strace -v ./out.log:/out.log /usr/bin/strace -s 9999 -fe execve,execveat,open,openat docker-entrypoint.sh mariadbd

Howerver, when i try running the a test version of the find locally (not in a docker container), I see clearly the execve call to chown.

$ cat test.sh
#!/bin/sh

for file in /.; do
        find $(dirname file) -type f -exec chown $USER:$USER {} \;
done

`

$ cat out.log
5147  execve("./test.sh", ["./test.sh"], 0x7fff932870b0 /* 18 vars */) = 0
5147  open("./test.sh", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
5148  execve("/usr/bin/dirname", ["dirname", "file"], 0x7f643fa58490 /* 18 vars */) = 0
5148  +++ exited with 0 +++
5147  --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=5148, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
5149  execve("/usr/bin/find", ["find", ".", "-type", "f", "-exec", "chown", "root:root", "{}", ";"], 0x7f643fa58618 /* 18 vars */) = 0
5149  open(".", O_RDONLY|O_LARGEFILE|O_CLOEXEC|O_DIRECTORY) = 3
5149  open("./.config", O_RDONLY|O_LARGEFILE|O_CLOEXEC|O_DIRECTORY) = 4
5149  open("./.config/micro", O_RDONLY|O_LARGEFILE|O_CLOEXEC|O_DIRECTORY) = 5
5150  execve("/usr/local/sbin/chown", ["chown", "root:root", "./.config/micro/settings.json"], 0x7ffe6cb2bbf8 /* 18 vars */) = -1 ENOENT (No such file or directory)
5150  execve("/usr/local/bin/chown", ["chown", "root:root", "./.config/micro/settings.json"], 0x7ffe6cb2bbf8 /* 18 vars */) = -1 ENOENT (No such file or directory)
5150  execve("/usr/sbin/chown", ["chown", "root:root", "./.config/micro/settings.json"], 0x7ffe6cb2bbf8 /* 18 vars */) = -1 ENOENT (No such file or directory)
5150  execve("/usr/bin/chown", ["chown", "root:root", "./.config/micro/settings.json"], 0x7ffe6cb2bbf8 /* 18 vars */) = -1 ENOENT (No such file or directory)
5150  execve("/sbin/chown", ["chown", "root:root", "./.config/micro/settings.json"], 0x7ffe6cb2bbf8 /* 18 vars */) = -1 ENOENT (No such file or directory)
5150  execve("/bin/chown", ["chown", "root:root", "./.config/micro/settings.json"], 0x7ffe6cb2bbf8 /* 18 vars */) = 0
5150  open("/etc/passwd", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
5150  open("/etc/group", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
5150  +++ exited with 0 +++
...

Do I need to add *CAP_SYS_PTRACE* to the container when running the strace probe or anything else?

So, I decided to start learn about system call with strace and want to observe network-related system call on apache2 processes, here's how I attach it:

pidof -s apache2
pstree -sTp 
strace -f -e trace=%network -p

and while observing, I notice that strace print some syscall, however I can't find the associated recvfrom or sendto syscall with file descriptor that correspond to the accept syscall in which contain the ip address of client (my browser) when I make a http request my assumption is that when a request is handled by apache, it spawn new processes as a child process, and since I attach the strace to the parent apache2 process, why the strace not follow its child despite I specify -f option?

I have an alias set for my rm command. If I run alias command, this is what I get as output. alias rm='rm -i' Now, when I run the rm command it works fine as expected. rm ramesh rm: remove regular empty file `ramesh'? y Now, I was learning on the system calls that are being called when I execute a command. For that I got to know about the strace command from here which lists me the files that are being called when I execute some command. The command is as below. strace -ff -e trace=file rm ramesh 2>&1 The command works perfectly fine except that it ignores my aliases that I have in place for my rm command. It deletes the file without prompting the user. So, does strace ignore aliases like this? If so why is it so? **EDIT:** Not sure, if this has something to do but type -a rm gives me the output as, rm is aliased to `rm -i' rm is /bin/rm So is it considering /bin/rm in this case which is why the user is not prompted before deletion?

I am very new to linux and as such would appreciate any pointers with respect to understanding system calls and having the ability, knowledge and tools to reverse-engineer their origin or their process flow. As the title suggests, i present an example, being my analysis of an Xorg process that i traced in my linux desktop environment. As such, i am attempting to understand the process flow of DRM_IOCTL calls, in this case a specific DRM_IOCTL_CURSOR2 system call that takes place within the process. My goal is to understand what triggers this call within this desktop environment, or rather what steps I can take in general to investigate inquiries like this From my limited understanding I am aware that Xorg is spawned as a subprocess of SDDM but aside from initiating the Xorg server, I am at a blank in trying to figure out how to walk through the process and identify triggers for certain process calls or perhaps the use of tools to do so. As such this is a conceptual question on how to approach analyses such as this in general. Would this require specific knowledge of the process at hand and its architecture. Would there be any general approaches I can take on my system to trace systemcalls much like deducing ppids of processes for my own interest. As of now I have vague familiarity using tools like strace, bpftrace and general command line tools like ps & lsof. Apologies if this is a broad question, if so I will be happy to narrow it further

OS is Debian 12. I copied some data to a usb drive, then ran the sync command to ensure I could remove it safely. The sync command then appeared to be stuck. After waiting for half an hour, I tried attaching strace, which showed no changing output. I cannot kill the sync, even with kill -9. ps shows the sync command in the "D" state. How do I get the sync command to exit without rebooting? What is wrong with sync? Is it just not used anymore? I've never been able to run it before without something like this happening. Can I just leave the sync stuck in "D" for now? Will it interfere with my normal desktop operation? I really do not want to reboot.

I have a file in $HOME/bin (before you ask, yes, it is in my path) called test which I've confirmed can be executed fine when I run it with the full path to the file. However, I get a really weird issue when I don't run it this way. When I just run test in the terminal, it doesn't do anything and returns immediately. I know that this isn't an issue of finding the file for several reasons: 1. There is no error message. Normally if the file can't be found or can't be executed a message will be printed out saying so. 2. Running which test still returns the correct file path. 3. Probably the weirdest of all - the script works fine when run through strace. I tried using strace to see if I could figure out what was going on but when I ran it with strace, it worked as expected with 0 issues.

I can run strace on a command like sleep 1 and see what files it's accessing like this: strace -e trace=file -o strace.log sleep 1 However, on my machine, many of the calls have a return value of -1 indicating that the file does not exist. For example: $ grep '= -1 ENOENT' strace.log | head access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/locale/en_US.UTF-8/LC_IDENTIFICATION", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/locale/en_US.UTF-8/LC_MEASUREMENT", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/locale/en_US.UTF-8/LC_TELEPHONE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/locale/en_US.UTF-8/LC_ADDRESS", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/locale/en_US.UTF-8/LC_NAME", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib/locale/en_US.UTF-8/LC_PAPER", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) I'm not really interested in the files that don't exist, I want to know what files the process actually found and read from. Aside from grep -v '=-1 ENOENT', how can I reliably filter out failed calls? # Addendum # I was surprised to learn that strace has had this feature in the works since 2002 in the form of the -z flag, which is an alias for -e status=successful, fully functional [since version 5.2](https://github.com/strace/strace/commit/e45a594cb08394c96f71105db9bacf08aa4c734d) ([2019-07-12](https://github.com/strace/strace/releases/tag/v5.2)) , also available as --successful-only [since version 5.6](https://github.com/strace/strace/commit/092724f8041cdfb64dcaf68a2d8ba877b509ea83) ([2020-04-07](https://github.com/strace/strace/releases/tag/v5.6)) . Also available since version 5.2 is the complement of -z, the -Z flag, which is an alias for -e status=failed, available as --failed-only since version 5.6. The -z flag was [first added in a commit from 2002](https://github.com/strace/strace/commit/17f8fb3484e94976882f65b7a3aaffc6f24cd75d) and released in version 4.5.18 ([2008-08-28](https://github.com/strace/strace/releases/tag/v4.5.18)) , bit it had never been [documented](https://github.com/strace/strace/commit/de6e53308ca58da7d357f8114afc74fff7a18043) because it was not working properly. Relevant links: - only seeing successful system calls Sat Nov 2 23:07:23 UTC 2002 > When using strace I sometimes like to see the system calls which work (instead of all the system calls). > > I've been porting this patch for years, it seems very useful. > > With the -z option, you don't see opens on files which aren't there (very useful tracking down what a program actually does, instead of trying to do). https://lists.strace.io/pipermail/strace-devel/2002-November/000232.html - strace: -z option doesn't work properly Date: Sun, 12 Jan 2003 09:33:01 UTC https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=176376 - tracing only failing syscalls Created: 2004-03-19 https://sourceforge.net/p/strace/feature-requests/3/ - [strace-4.15] Proposal: Output Staging for -z Option (print successful syscalls only) / Patch included Tue Jan 17 09:35:54 UTC 2017 https://lists.strace.io/pipermail/strace-devel/2017-January/005941.html - [PATCH v1] Implemented output staging for failed/successful syscalls Wed Jan 18 16:01:20 UTC 2017 https://lists.strace.io/pipermail/strace-devel/2017-January/005950.html - Fix -z option Feb 28, 2018 https://github.com/strace/strace/issues/49 - [PATCH 0/3] Stage output for -z and new -Z options Mon Apr 1 21:13:02 UTC 2019 https://lists.strace.io/pipermail/strace-devel/2019-April/008706.html - strace -z flag Mon Jun 10 05:29:19 UTC 2019 https://lists.strace.io/pipermail/strace-devel/2019-June/008808.html

The --vm flag of stress command says that, > -m, --vm N spawn N workers spinning on malloc()/free() I wanted to see what it is actually doing so I ran this command, strace stress --vm 1 I get the following output, execve("/bin/stress", ["stress", "--vm", "1"], [/* 27 vars */]) = 0 brk(NULL) = 0x55f05bef2000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fce9f15a000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=40388, ...}) = 0 mmap(NULL, 40388, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fce9f150000 close(3) = 0 open("/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0pS\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1139680, ...}) = 0 mmap(NULL, 3150136, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fce9ec38000 mprotect(0x7fce9ed39000, 2093056, PROT_NONE) = 0 mmap(0x7fce9ef38000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x100000) = 0x7fce9ef38000 close(3) = 0 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P%\2\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=2173512, ...}) = 0 mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fce9e86b000 mprotect(0x7fce9ea2e000, 2093056, PROT_NONE) = 0 mmap(0x7fce9ec2d000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c2000) = 0x7fce9ec2d000 mmap(0x7fce9ec33000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fce9ec33000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fce9f14f000 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fce9f14d000 arch_prctl(ARCH_SET_FS, 0x7fce9f14d740) = 0 mprotect(0x7fce9ec2d000, 16384, PROT_READ) = 0 mprotect(0x7fce9ef38000, 4096, PROT_READ) = 0 mprotect(0x55f05abdb000, 4096, PROT_READ) = 0 mprotect(0x7fce9f15b000, 4096, PROT_READ) = 0 munmap(0x7fce9f150000, 40388) = 0 getpid() = 5305 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fce9f159000 write(1, "stress: info: dispatching"..., 64stress: info: dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd ) = 64 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fce9f14da10) = 5306 wait4(-1 The output just stuck here and I don't see any malloc() calls either. How can I see it making malloc() and free() systemcalls?

The problem: **typing in the terminal on Ubuntu takes seconds until the typed character appears.** I first noticed this after I moved large chunks of data onto another LUKS encrypted HDD which I mounted via /etc/fstab automatically. Typing in the terminal (I'm using the default bash) responded uncannily slowly. First I thought that the problem is the additional HDD, but after I reinstalled Ubuntu on a new SSD, the problem kept coming back, but I did not mount the HDD at all. I don't think it's the hardware, as the other OS does not have any problem (dual boot on the Samsung 990 Pro). **Details:** * Fresh install of Ubuntu 22.04.4 LTS, full disk encryption with LUKS on a Samsung 870 QVO 4TB Sata 6Gb/s * Also reproducible on a Samsung 990 Pro 4TB M.2 vMVE (2 TB partition without encryption) * 4x8GB of RAM @2133 MT/s, at least 15GB free. * CPU: i7-6700K, all 8 cores read(0, "\33", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, "[", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, "A", 1) = 1 write(2, "echo $$", 7) = 7 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, "\33", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, "[", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, "A", 1) = 1 write(2, "\10\10\10\10\10\10\10cat /proc/sys/fs/inotify/max_user_watches", 48) = 48 What puzzles me is the **first pselect6 call, which runs for over a second**. That's definitely the culprit, but I don't know how to debug the problem from here. Here's another strace where I'm typing only spaces: strace: Process 6673 attached pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, " ", 1) = 1 pselect6(1, , NULL, , {tv_sec=0, tv_nsec=0}, NULL) = 0 (Timeout) write(2, " ", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, " ", 1) = 1 pselect6(1, , NULL, , {tv_sec=0, tv_nsec=0}, NULL) = 0 (Timeout) write(2, " ", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, " ", 1) = 1 pselect6(1, , NULL, , {tv_sec=0, tv_nsec=0}, NULL) = 0 (Timeout) write(2, " ", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, " ", 1) = 1 pselect6(1, , NULL, , {tv_sec=0, tv_nsec=0}, NULL) = 0 (Timeout) write(2, " ", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}) = 1 (in ) read(0, " ", 1) = 1 pselect6(1, , NULL, , {tv_sec=0, tv_nsec=0}, NULL) = 0 (Timeout) write(2, " ", 1) = 1 pselect6(1, , NULL, NULL, NULL, {sigmask=[], sigsetsize=8}strace: Process 6673 detached Sometimes pselect6 is really slow with 2.2 seconds, sometimes it's fast.

I'm trying to monitor read activity on a particular file descriptor in a running process. Here is the C++ test bench I'm testing various solutions on:

#include 
#include 
#include 
#include 

int main()
{
	const int fd1=open("/tmp/testfile", O_RDWR|O_CREAT);
	const int fd2=open("/tmp/testfile", O_RDONLY);
	write(fd1, "Hello, world!", 13);
	std::cout

But strace manual says about -e read=set: > Note that this is independent from the normal tracing of the read system call which is controlled by the option -e trace=read. So I suppose that my trace=none, which is supposed to avoid tracing anything unrelated to my read=3 option is correct. Apparently, I'm doing something wrong. So, how should I actually use strace to trace only the read system call, and only when fd==3? Note that filtering output using grep is not an option, since it'd slow down a real-world application I'm going to trace.

this makes vim madness: $strace -o >(vim -; stty sane) file.out; stty sane I have typed stty sane in processed substitution as well as in next command but none of them did their job. Once i piped the strace command to vim, then the vim no longer behaves normaly (I know vim can only accepts stdin and strace gives output to stderr, but then what is the -o flag for?) any help?

(I think) I have a relatively good understanding of how Linux permissions work - traditional Unix DAC permissions, and how they're represented, security context in relation to processes (i.e. cred struct, capabilities, etc.), hooks for LSMs, etc., and the link they have to (E)UID/E(GID), etc. What I don't understand, and am having difficulty finding in kernel code, and from using

(via

-cmd

), is when these checks are actually done (presumably) in kernel space, and at *which point* the access decision is actually made. To elaborate: I have a root-owned directory

/dir

chmod-ed to

-x------

(*500*). As an unprivileged user, I run

/bin/ls /dir

and receive a permission denied error, as expected. If I run

-s 10000 /bin/ls /dir

I can see the

syscall, which returns a

-1

back to user space, and sets

to

(i.e. "permission denied"). There's then a subsequent

syscall to print the error message to STDERR, all as expected. At no point here do I ever see

or

syscalls being called. If I run the same test using trace-cmd, as

-cmd record -p function_graph -F /bin/ls

with the same unprivileged user doing

/bin/ls /dir

in a separate terminal tab (where

-cmd

will trigger on the execution of the

binary), I can basically map most of the syscalls I'm seeing, one-to-one, between strace and trace-cmd outputs. However, the discrepancy is that, in the trace-cmd output after the

syscall, rather than seeing the write outputs and it exiting, I can see

calls being made (and the full call tree of iterating over items in the directory, etc.). Because of limitations with ftrace, I guess (and more likely limitations in my understanding), I'm not seeing syscall arguments or returns in the output, so I can't see when, if at all, I'm getting permission errors, but from my limited understanding, the kernel appears to be actually doing the full directory listing, but just not returning the output back to user space. So can anyone explain what's actually happening - why

-cmd

shows the kernel actually doing the

syscalls, while

exits after the open call fails? Most of the forum posts (on here and elsewhere) with questions pertaining to permissions suggest that checks are done on opening the file, but provide little more detail than that, though this aligns with what I can see with

(but not

/

-cmd

). Looking at the kernel source, and following that through manually from the

/

syscalls, I can see functions call sequences such as

-> do_inode_permission -> generic_permission -> acl_permission_check

with the latter actually doing comparisons on the classic Unix permissions, and returning from this. I can also see the hooks elsewhere for the LSMs, like SELinux, Apparmor, SMACK, Tomoyo, etc., which I guess will do their bits. My assumption would be that the directory is attempted to be opened, permission checks would happen, kernel says no, and returns the error back to user space, and it would never get to the point of actually doing the directory listing. However, based on the

-cmd

output appearing to actually show the directory listing happening, I'm not sure any more. Any information - in as much technical depth as possible - would be greatly appreciated. Also, I understand that using Systemtap may be able to offer more information to me, but my knowledge of that is *very* limited! For awareness, my testing has been on Kernel versions 2.6.35 and 6.5.0 on Debian-based distros, with comparatively similar results.

I want to trace the networking activity of a command, I tried tcpdump and strace without success. For an example, If I am installing a package or using any command that tries to reach some site, I want to view that networking activity (the site it tries to reach). I guess we can do this by using tcpdump. I tried but it is tracking all the networking activity of my system. Let's say if I run multiple networking related commmands and I want to track only particular command networking activity, that time it is difficult to find out the exact solution. Is there a way to do that? **UPDATE:** I don't want to track everything that goes on my network interface. I just want to track the command (for an example #yum install -y vim) networking activity. Such as the site it tries to reach.

I use strace to trace the behavior of a bash process. The purpose is to find out the order bash loads its configuration files. I am running the following command under zsh:

strace -e openat bash

After running this command, I end up in a new bash shell, but I don't want that to happen. Is there any way to trace the bash command without actually starting new bash interactive shell? I searched online but couldn't find anything. I was trying this with exec: strace -e openat "$(exec bash)" 2>&1, but still my shell changes to bash from zsh.

I'm trying to write the output of strace ls to a file. I know that I need to use > in order to forward output of a command to a file, but it doesn't work. It creates a file but the command prints the output of strace ls to stdout but writes the file name into the file.

$ strace ls > ls_sys.txt
...
strace output
...
$ cat ls_sys.txt
ls_sys.txt

I strace a curl command:

strace -s 2000 -f curl google.com

and see 2 DNS queries

recvfrom(3, "\302\325\201\200\0\1\0\1\0\0\0\0\6google\3com\0\0\34\0\1\6google\3com\0\0\34\0\1\0\0\0\362\0\20*\0\24P@\t\10\v\0\0\0\0\0\0 \16", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.65.7")}, [28->16]) = 66
recvfrom(3, "X\320\201\200\0\1\0\1\0\0\0\0\6google\3com\0\0\1\0\1\6google\3com\0\0\1\0\1\0\0\1)\0\4\216\372\263\356", 65536, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.65.7")}, [28->16]) = 54

Then, I see a connect() syscall to 142.250.179.238 which is a Google's IP.

connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("142.250.179.238")}, 16) = 0

I believe one of the 2 recvfrom calls contains the IP "142.250.179.238", otherwise curl can't know the IP to connect to. **My question: What is the format of the 2 recvfrom strings? And how to parse it to get the IP address?**

"\302\325\201\200\0\1\0\1\0\0\0\0\6google\3com\0\0\34\0\1\6google\3com\0\0\34\0\1\0\0\0\362\0\20*\0\24P@\t\10\v\0\0\0\0\0\0 \16"
"X\320\201\200\0\1\0\1\0\0\0\0\6google\3com\0\0\1\0\1\6google\3com\0\0\1\0\1\0\0\1)\0\4\216\372\263\356"

I am using slim login manager. when I log out from my session, or restart slim from the commandline: service slim restart I get blank console for couple of minutes, before the slim login screen appears again. When I attach to the slim process with strace, I see many "Resource temporarily unavailable" and "Connection timed out" lines: socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 4 connect(4, {sa_family=AF_UNIX, sun_path=@"/tmp/.X11-unix/X0"}, 20) = -1 ECONNREFUSED (Connection refused) close(4) = 0 socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 4 getsockopt(4, SOL_SOCKET, SO_SNDBUF, , ) = 0 connect(4, {sa_family=AF_UNIX, sun_path="/tmp/.X11-unix/X0"}, 110) = -1 ENOENT (No such file or directory) close(4) connect(4, {sa_family=AF_INET, sin_port=htons(6000), sin_addr=inet_addr("127.0.0.1")}, 16) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=5440, si_uid=0} --- rt_sigaction(SIGUSR1, {sa_handler=0x55961af808a0, sa_mask=[USR1], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f545cc5afd0}, {sa_handler=0x55961af808a0, sa_mask=[USR1], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f545cc5afd0}, 8) = 0 rt_sigreturn({mask=[]}) = 42 connect(4, {sa_family=AF_INET, sin_port=htons(6000), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ETIMEDOUT (Connection timed out) close(4) full strace here after few minutes slim login screen reappears again, and everything works fine. When I restart computer, slim works fine the first time, probably because I have autologin in my /etc/slim.conf: auto_login yes What is going on? And how can I fix it?