What exactly is happening here? root@bob-p7-1298c:/# ls -l /tmp/report.csv && lsof | grep "report.csv" -rw-r--r-- 1 mysql mysql 1430 Dec 4 12:34 /tmp/report.csv lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs Output information may be incomplete.

$ docker run --rm nginx:alpine
$ docker exec --privileged -it docker ps -q | head -1 sh
/ # apk add strace lsof
/ # strace -fp1 lsof
strace: Process 1 attached
[pid     1] rt_sigsuspend([], 8 
[pid   100] execve("/usr/bin/lsof", ["lsof"], 0x7fff9ca1d308 /* 9 vars */) = 0
[pid   100] arch_prctl(ARCH_SET_FS, 0x7f19d2cc8b48) = 0
[pid   100] set_tid_address(0x7f19d2cc8fb0) = 100
[pid   100] brk(NULL)                   = 0x55791ae48000
[pid   100] brk(0x55791ae4a000)         = 0x55791ae4a000
[pid   100] mmap(0x55791ae48000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x55791ae48000
[pid   100] mprotect(0x7f19d2cc5000, 4096, PROT_READ) = 0
[pid   100] mprotect(0x557919b6d000, 4096, PROT_READ) = 0
[pid   100] prlimit64(0, RLIMIT_NOFILE, NULL, {rlim_cur=1073741816, rlim_max=1073741816}) = 0
[pid   100] close_range(3, 1073741815, 0) = 0
[pid   100] open("/dev/null", O_RDWR|O_LARGEFILE) = 3
[pid   100] close(3)                    = 0
[pid   100] umask(000)                  = 022
[pid   100] getpid()                    = 100
[pid   100] getgid()                    = 0
[pid   100] getegid()                   = 0
[pid   100] geteuid()                   = 0
[pid   100] getuid()                    = 0
[pid   100] mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f19d2c2b000
[pid   100] stat("/dev", {st_mode=S_IFDIR|0755, st_size=340, ...}) = 0
[pid   100] open("/", O_RDONLY|O_LARGEFILE) = 3
[pid   100] lseek(3, 1, SEEK_SET)       = 1
[pid   100] lstat("/proc/100/fd/3", {st_mode=S_IFLNK|0500, st_size=64, ...}) = 0
[pid   100] open("/proc/100/fdinfo/3", O_RDONLY|O_LARGEFILE) = 4
[pid   100] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f19d2c2a000
[pid   100] read(4, "pos:\t1\nflags:\t0100000\nmnt_id:\t35"..., 1024) = 47
[pid   100] lseek(4, -40, SEEK_CUR)     = 7
[pid   100] close(4)                    = 0
[pid   100] munmap(0x7f19d2c2a000, 4096) = 0
[pid   100] close(3)                    = 0
[pid   100] open("/proc/mounts", O_RDONLY|O_LARGEFILE) = 3
[pid   100] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f19d2c2a000
[pid   100] read(3, "overlay / overlay rw,relatime,lo"..., 4088) = 2115
[pid   100] pipe([4, 5])                = 0
[pid   100] pipe([6, 7])                = 0
[pid   100] rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
[pid   100] rt_sigprocmask(SIG_BLOCK, ~[], ~[KILL STOP RTMIN RT_1 RT_2], 8) = 0
[pid   100] fork(strace: Process 101 attached
)                      = 101
[pid   101] gettid()                    = 101
[pid   100] rt_sigprocmask(SIG_SETMASK, ~[KILL STOP RTMIN RT_1 RT_2],  
[pid   101] rt_sigprocmask(SIG_SETMASK, ~[KILL STOP RTMIN RT_1 RT_2],  
[pid   100] NULL, 8) = 0
[pid   101] NULL, 8) = 0
[pid   100] rt_sigprocmask(SIG_SETMASK, [],  
[pid   101] rt_sigprocmask(SIG_SETMASK, [],  
[pid   100] NULL, 8) = 0
[pid   101] NULL, 8) = 0
[pid   100] close(4 
[pid   101] close(0 
[pid   100] )        = 0
[pid   101] )        = 0
[pid   100] close(7 
[pid   101] close(1 
[pid   100] )        = 0
[pid   101] )        = 0
[pid   100] rt_sigprocmask(SIG_UNBLOCK, [RT_1 RT_2],  
[pid   101] close(2 
[pid   100] NULL, 8) = 0
[pid   101] )        = 0
[pid   100] rt_sigaction(SIGALRM, {sa_handler=0x557919b5f4b6, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f19d2c78acd},  
[pid   101] close(3 
[pid   100] {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
[pid   101] )        = 0
[pid   100] setitimer(ITIMER_REAL, {it_interval={tv_sec=0, tv_usec=0}, it_value={tv_sec=15, tv_usec=0}},  
[pid   101] close(5 
[pid   100] {it_interval={tv_sec=0, tv_usec=0}, it_value={tv_sec=0, tv_usec=0}}) = 0
[pid   101] )        = 0
[pid   100] write(5, "\315\364\265\31yU\0\0", 8 
[pid   101] close(6 
[pid   100] )        = 8
[pid   101] )        = 0
[pid   100] write(5, "\2\0\0\0", 4 
[pid   101] close(8 
[pid   100] )        = 4
[pid   101] )        = -1 EBADF (Bad file descriptor)
[pid   100] write(5, "/\0", 2 
[pid   101] close(9 
[pid   100] )        = 2
[pid   101] )        = -1 EBADF (Bad file descriptor)
[pid   100] write(5, "\0\20\0\0", 4 
[pid   101] close(10 
[pid   100] )        = 4
[pid   101] )        = -1 EBADF (Bad file descriptor)
[pid   100] read(6,  
[pid   101] close(11)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(12)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(13)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(14)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(15)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(16)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(17)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(18)                   = -1 EBADF (Bad file descriptor)
[pid   101] close(19)                   = -1 EBADF (Bad file descriptor)
...
[pid   101] close(1141630)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141631)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141632)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141633)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141634)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141635)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141636)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141637)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141638)              = -1 EBADF (Bad file descriptor)
[pid   101] close(1141639strace: Process 1 detached
strace: Process 100 detached
strace: Process 101 detached

Is there a way to make it work?

I am trying to find out if lsof is part of a typical RHEL installation. I see lsof mentioned in a RHEL page titled Common administrative commands in Red Hat Enterprise Linux 5, 6, 7, and 8 but I do not know enough about Linux administration to know whether this means it will always be there or that it may need to be installed. I am trying to find this out so I can decide if I need to advise uses to install it, or if they (and my script) can assume it will be there. On a related note, how do I find out everything that a distro would have?

I know I can view the open files of a process using lsof *at that moment in time* on my Linux machine. However, a process can open, alter and close a file so quickly that I won't be able to see it when monitoring it using standard shell scripting (e.g. watch) as explained in ["monitor open process files on linux (real-time)"](https://serverfault.com/questions/219323/monitor-open-process-files-on-linux-real-time) . So, I think I'm looking for a simple way of auditing a process and see what it has done over the time passed. It would be great if it's also possible to see what network connections it (tried to) make and to have the audit start before the process got time to run without the audit being started. Ideally, I would like to do this: sh $ audit-lsof /path/to/executable 4530.848254 OPEN read /etc/myconfig 4530.848260 OPEN write /var/log/mylog.log 4540.345986 OPEN read /home/gert/.ssh/id_rsa 1.2.3.4:80 | [...] 4541.023485 CLOSE /home/gert/.ssh/id_rsa 1.2.3.4:80 | this when polling Would this be possible using strace and some flags to not see every system call?

I'm working an issue with [ZeroMQ](https://zeromq.org/) using [XSub-XPub](https://netmq.readthedocs.io/en/latest/xpub-xsub/) . I did lsof on bind in port and see:

lsof -i :5556
COMMAND    PID      USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
python  475590 codespace   45u  IPv4 1432077      0t0  TCP localhost:51620->localhost:freeciv (ESTABLISHED)
python  475684 codespace   12u  IPv4 1421144      0t0  TCP *:freeciv (LISTEN)
python  475684 codespace   15u  IPv4 1430650      0t0  TCP localhost:freeciv->localhost:51620 (ESTABLISHED)

What is freeciv? Search only yields [a video game](https://freeciv.org/) , but it's not that. Ubuntu 20.04.6.

I keep having

lsof -w -l +d /var/lib/php5

eating up my CPU I want to know who is triggering it and what does it have with php5 ...

OS is Debian. I'm running nginx as a webserver. I am not running Wordpress. Logging is enabled in the http block with: access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; Seeing hundreds of connections in iftop from my webserver at port 443 to 191-242.x.x.alivenet.com.br, for a few hundred different ips. The strange things are: * None of the ip addresses listed by iftop show up in my nginx logs, which I have been retaining since server creation. * These connections show up in iftop even if nginx isn't running. How is that possible? I tried stopping nginx, then confirmed with netstat that nothing is listening on ports 80 or 443, but I still see hundreds of these connections listed in iftop. How can iftop show these connections if nothing is listening on those ports? I even tried disabling nginx and then rebooting my server, but they still show up. * lsof -a -i4 -i6 -itcp doesn't show any of these connections somehow. * In nethogs, I see a line: ? root :443-191.242.x.x:. Running as root? Question mark for the pid? This seems absolutely crazy to me. Does this mean there is some process running on my system that somehow isn't assigned a pid, running as root, somehow listening on port 443 despite nginx supposedly currently using that port, that's sending traffic to one of these Brazilian ips? What exactly is going on here? Do I need to be concerned? Has my server been hacked? What else should I check to confirm the server is ok? Do I need to block these ips via iptables? How do I separate hack attempts from legitimate web server traffic? (ie, someone in Brazil is trying to visit my website) How can nethogs not show a pid? How can iftop show connetions on ports that aren't listening? How can these connections not show up in the nginx logs?

Is there a way to make lsof work continuously to monitor every file that is being opened in real time? I don't know the name of the process. I want lsof to work continuously for a period of time until I see the the list contains what I want.

I do not understand the output of the lsof command. When I write: lsof -p PID I get 4 lines, and when I write: lsof | grep PID I get hundreds of lines. Shouldn't it return the same result? Here are some sample outputs. Looks like it is a subprocess, or what do those tasks mean? lsof -p 29076 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 29076 pr cwd unknown /proc/29076/cwd (readlink: Permission denied) java 29076 pr rtd unknown /proc/29076/root (readlink: Permission denied) java 29076 pr txt unknown /proc/29076/exe (readlink: Permission denied) java 29076 pr NOFD /proc/29076/fd (opendir: Permission denied) lsof |grep 29076|head -20 java 29076 pr cwd unknown /proc/29076/cwd (readlink: Permission denied) java 29076 pr rtd unknown /proc/29076/root (readlink: Permission denied) java 29076 pr txt unknown /proc/29076/exe (readlink: Permission denied) java 29076 pr NOFD /proc/29076/fd (opendir: Permission denied) java 29076 300 pr cwd unknown /proc/29076/task/300/cwd (readlink: Permission denied) java 29076 300 pr rtd unknown /proc/29076/task/300/root (readlink: Permission denied) java 29076 300 pr txt unknown /proc/29076/task/300/exe (readlink: Permission denied) java 29076 300 pr NOFD /proc/29076/task/300/fd (opendir: Permission denied) java 29076 329 pr cwd unknown /proc/29076/task/329/cwd (readlink: Permission denied) java 29076 329 pr rtd unknown /proc/29076/task/329/root (readlink: Permission denied) java 29076 329 pr txt unknown /proc/29076/task/329/exe (readlink: Permission denied) java 29076 329 pr NOFD /proc/29076/task/329/fd (opendir: Permission denied) java 29076 330 pr cwd unknown /proc/29076/task/330/cwd (readlink: Permission denied) java 29076 330 pr rtd unknown /proc/29076/task/330/root (readlink: Permission denied) java 29076 330 pr txt unknown /proc/29076/task/330/exe (readlink: Permission denied) java 29076 330 pr NOFD /proc/29076/task/330/fd (opendir: Permission denied) java 29076 331 pr cwd unknown /proc/29076/task/331/cwd (readlink: Permission denied) java 29076 331 pr rtd unknown /proc/29076/task/331/root (readlink: Permission denied) java 29076 331 pr txt unknown /proc/29076/task/331/exe (readlink: Permission denied) java 29076 331 pr NOFD /proc/29076/task/331/fd (opendir: Permission denied)

lsof -p PID works fine to list all files open by a single process, but I need to list all currently open files by a specific master process AND its children processes. What is the best way to do this? If this isn't possible with lsof, what other way could I accomplish this?

I want to remove my external HDD as safely as possible. I want to use umount --lazy: > Lazy unmount. Detach the filesystem from the file hierarchy now, and clean up all references to this filesystem as soon as it is not busy anymore. (Requires kernel 2.4.11 or later.) Then after a short delay I plan to kill any processes with open files on the device where the filesystem is still quasi-mounted. * I can't use lsof for an accurate list of the open files as the filesystem has become invisible to new processes. * If I use lsof before umount -l, there is a race contition of a new file being opened in between the two invocations. Is there any way of finding out which processes are accessing a DEVICE rather than a filesystem?

OS is Debian. I have a few systems that seem to not be freeing disk space correctly. On a **normal** system: du --max-depth 1 / --exclude=/proc -h -x Seems to almost exactly agree with: df -h / ie, ~800 gigs are reported used in both cases for my normal system test case. On an **affected** system, df shows 95 gigs used, but du only shows ~55 gigs used! This is a huge discrepancy, especially compounded by the fact that the affected system also reports 0 bytes available for /. lsof | grep deleted shows nothing. Filesystem is ext4. What is going on here? Edit: The issue ended up being a problem with lsof. To briefly summarize: running lsof without any parameters ended up taking so long (hours) that it appeared that nothing was getting returned, but I think lsof was just getting killed. Running lsof like this: lsof / | grep deleted was much, much faster, would exit correctly, and when doing so would show one very large deleted file. Running lsof / also excludes any other mount points, even if they are "inside" of /.

In my example, the wc program is trying to open the test FIFO or named pipe. These in-progress open syscalls seem not to be shown by fuser or lsof:

mknod /tmp/testpipe p
wc /tmp/testpipe &
timeout 0.2 strace -p $! |& timeout 0.1 cat; echo
strace: Process 10103 attached
open("/tmp/testpipe", O_RDONLY

fuser /tmp/testpipe  # no output
lsof | grep testpipe  # no output

How to find processes trying to open some FIFO in Linux systems ?

When I try to remount a partition as read-only, I get an error /foo is busy. I can list all files open from /foo with lsof /foo but that does not show me whether files are open read-only or read-write. Is there any way to list only files which are open as read-write ?

When I use ss (socket statistics) to show the usages of port 5432 I get:

$ sudo ss -ln | grep -E 'State|5432'
Netid State  Recv-Q Send-Q                     Local Address:Port    Peer Address:PortProcess
u_str LISTEN 0      244    /var/run/postgresql/.s.PGSQL.5432 54481              * 0
tcp   LISTEN 0      244                            127.0.0.1:5432         0.0.0.0:*

When using lsof (list of open files) instead I get no result:

$ sudo lsof -i tcp:5432

Why is that? Related to: - https://unix.stackexchange.com/q/652556/87249 - [Difference between lsof -i : & socket statistics ss -lp | grep ?](https://stackoverflow.com/q/77685954/334569) **Edit with answers from comments:** - sudo ss -lnp does not show the pid of the process(es) that have that listening socket - the 127.0.0.1:5432 0.0.0.0:* on the last line was a copy-paste error, sorry about that, I have removed it - I am running those commands in a WSL terminal, Postgres is not running anywhere **Edit with new findings:** I have found out this is happening only when Docker Desktop is running (even though there is no container running): ss doesn't output anything once I quit Docker Desktop. It looks like this might be an issue somehow related with Docker Desktop: I have reported it in this GitHub issue.

On our A100 machine, we frequently have zombie processes that still allocate memory when already stopped. I usually used fuser -v /dev/nvidia* to determine the PIDs of all processes and kill them either with kill or fuser -k /dev/nvidia*. fuser always took a short while to return the result. But now, the command fuser -v and fuser -k are hanging indefinitely, not responding within any reasonable amount of time. For example, last time, it ran a weekend without returning. I ended up restarting the server. fuser -v /dev/nvidia0 shows the same abnormal behavior, as does lsof /dev/nvidia0. When looking online for this issue, I only get answers to the zombie process problem mentioned above, but no problems specifically for the case when fuser or lsof are hanging. How to debug/solve this problem ideally without restarting the machine? The system runs Ubuntu 20.04.

I have the following command that works great: sudo lsof -w -c smbd -u myuser | grep '.txt' I just want to get the full path to the file name part of the returned output, the "NAME" column, to put in a report. What is the easiest way to just grab the NAME column?

lsof is a great utility, just now started using it. lsof -i | grep smtp => this give the following result. httpd.pl 212548 global 3u IPv4 893092369 0t0 TCP server07.host...blah... In the above example, httpd.pl is perl script, which sends spam emails. **How can I know the full path of the command. i.e in the above result**, I want to know the full path of httpd.pl I tried searching in home dir, the file httpd.pl is not there. and also, I tried with lsof -p PID, this also does not give the path to it. Is there any way, I can get the full path of that file ? Note: This kind of problem is very common in shared hosting environment. So, it will be very useful for shared hosting or any web server administrator.

When I run lsof I see the output

java      1752736                               user 9995u     sock                0,8       0t0 1432559505 protocol: TCPv6
java      1752736                               user *527u     sock                0,8       0t0 1444900878 protocol: TCPv6

What does the * in front of the file descriptor indicate?

I am using this command below and trying to separate the columns I only want to get the PID to use it in my python script. I can easily get this line by line but then how to separate into columns in a non hacky way? I can easily split by space but lets face it that is a terrible idea, any suggestions? root@python-VirtualBox:/var/python# lsof | grep TCP lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs Output information may be incomplete. COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME sshd 3449 root 3u IPv4 24248 0t0 TCP *:22 (LISTEN) sshd 3449 root 4u IPv6 24257 0t0 TCP *:22 (LISTEN)