I normally watch many logs in a directory doing tail -f directory/*. The problem is that a new log is created after that, it will not show in the screen (because * was expanded already). Is there a way to monitor every file in a directory, even those that are created after the process has started?

The Logstash documentation said about the input file plugin that it > Stream events from files, normally by tailing them in a manner similar to tail -0F but optionally reading them from the beginning. The tail man page said that -f, --follow[={name|descriptor}] output appended data as the file grows; an absent option argument means 'descriptor' -F same as --follow=name --retry But what is exactly the -0 flag in tail ?

I docker that output logs stdout stderr, which can be viewed using: docker logs -f $LOGS_CONTAINER_ID I also added 'sed', which puts the container id on each line: docker logs -f $LOGS_CONTAINER_ID | sed "s/^/$LOGS_CONTAINER_ID /" If I run it, I get something like: container112 error 10:20:10 problem container112 info 10:20:09 not problem container112 error 10:20:01 problem where "container112" is $LOGS_CONTAINER_ID. SO FAR SO GOOD. Now I want to output the above command to a file (log.out), so I wrote the following command: docker logs -f $LOGS_CONTAINER_ID | sed "s/^/$LOGS_CONTAINER_ID /" >> log.out What happens is that it writes the logs to log.out, but it doesn't get new logs (if I open a new session and run tail -f log.out, I don't get output). So I also tried: tail -f $(docker logs -f $LOGS_CONTAINER_ID | sed "s/^/$LOGS_CONTAINER_ID /") >> log.out But it also didn't work. What is the problem?

I'd like to be able to tail the output of a server log file that has messages like: INFO SEVERE etc, and if it's SEVERE, show the line in red; if it's INFO, in green. What kind of alias can I setup for a tail command that would help me do this?

I observe that, on Ubuntu 24.04.2 with coreutils version 9.4-3ubuntu6, running:

bash
$ tail -c 4097 /dev/zero
$ echo $?
0

exits immediately with a status code of 0. I expected the command to block indefinitely since /dev/zero is an endless stream. In contrast, the following commands behave as expected (i.e., they block until interrupted):

bash
$ tail -c 4096 /dev/zero
^C
$ echo $?
130
$ cat /dev/zero | tail -c 4097
^C
$ echo $?
130

## Debug attempt The strace output shows differences between the two invocations: | strace tail -c 4096 /dev/zero | strace tail -c 4097 /dev/zero | |---------------------------------------------------------------------|---------------------------------------------------------------------| | … | … | | close(3) = 0 | close(3) = 0 | | openat(AT_FDCWD, "/dev/zero", O_RDONLY) = 3 | openat(AT_FDCWD, "/dev/zero", O_RDONLY) = 3 | | fstat(3, {st_mode=S_IFCHR\|0666, st_rdev=makedev(0x1, 0x5), …}) = 0 | fstat(3, {st_mode=S_IFCHR\|0666, st_rdev=makedev(0x1, 0x5), …}) = 0 | | lseek(3, -4096, SEEK_END) = 0 | lseek(3, -4097, SEEK_END) = 0 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | read(3, "\0\0\0\0\0\0\0\0\0\\…, 4097) = 4097 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | fstat(1, {st_mode=S_IFIFO\|0600, st_size=0, …}) = 0 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | write(1, "\0\0\0\0\0\0\0\0\0\\…, 4096 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | close(3) = 0 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | write(1, "\0", 1) = 1 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | close(1) = 0 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | close(2) = 0 | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | exit_group(0) = ? | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | ~~+~~ exited with 0 ~~+~~ | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | | | read(3, "\0\0\0\0\0\0\0\0\00"…, 8192) = 8192 | | | … | |

I'm working with a directory of multiple log files. I'm trying to show a running count, updated periodically, of the number of lines written to each log file. I need to see the number of lines written per period (e.g. per second) to each file, indexed by file name. For bonus points, the lines should be sorted in decreasing order of the number of lines written per second. This doesn't have to use something like ncurses (e.g. like top); it's ok if it simply writes updates every period (e.g. every second). How do I accomplish this task? --- I've included my solution as an answer, but I suspect there's a better way to do this... Hoping to learn a better way!

I'm usually inside GNU Screen or tmux, and that doesn't give me great scrolling functionality. Is there an alternative to tail -f that allows me to quickly scroll up? A tool that is like most is to less and more. *[This question](https://unix.stackexchange.com/questions/21808/tail-f-equivalent) is related but far from specific. I'm really looking for something that lets me scroll.*

I'm tailing logs of my own app and Postgres. tail -f /tmp/myapp.log /var/log/postgresql/postgresql.main.log I need to include pgpool 's logs. It used to be syslog, but now it is in journalctl. Is there a way to tie tail -f && journalctl -f together?

I'm using **less** to continuously trace Squid log file (as well as UFW log) with this command: less --follow-name -K +F /var/log/squid/access.log And at the time of rotation of Squid log **less** quits. I guess this happens because when an old file is renamed the new one is not created immediately but with a delay, though in the case of UFW log file this doesn't happen and **less** successfully switches to the new file. So is there a method or option to make **less** wait for a new file to appear?

I have a server log that outputs a specific line of text into its log file when the server is up. I want to execute a command once the server is up, and hence do something like the following: tail -f /path/to/serverLog | grep "server is up" ...(now, e.g., wget on server)? What is the best way to do this?

I have a file with about 30.000.000 lines (Radius Accounting) and I need to find the last match of a given pattern. The command: tac accounting.log | grep $pattern gives what I need, but it's too slow because the OS has to first read the whole file and then send to the pipe. So, I need something fast that can read the file from the last line to the first.

Due to my lack of experience with script language, (shame for a Mac user) I have referred to several sources: link seemed resolved with ls *.extension | xargs -n 1 tail -n +2 This didn't for me, even after adding > merged.txt at the end, nor the following: for f in *.txt do tail -n +2 $f >> /path/to/some/dir/with/files/file_name done I also tried sed -e'1d' $FILE in replacement of the tail command. Didn't work. tail -n +2 file_name.extension, cat LIN_1994-11_0100.txt | tail -n +2, awk 'FNR != 1' *.extension has no effect to the file. I am uncertain if this has anything to do with the current issue. Or whether the link is related to the issue. If anyone could find the reason for this problem or way out of it..would be majorly grateful. I have transferred this issue from another community here to receive more insight if I could.

How to I continuously tail -f a log, find all files (sed), and display (cat) the found files ## example data in audit logs. tail -f /var/log/httpd/modsec_audit.log | sed 's/[^\/]*/\./;s/].*$//g' ### output ./apache/20180508/20180508-1428/20180508-142802-WvH6QgoeANwAAMwsFZ4AAAAF ./apache/20180508/20180508-1428/20180508-142803-WvH6QgoeANwAAMwtFfcAAAAG ./apache/20180508/20180508-1428/20180508-142803-WvH6QwoeANwAAMwuFlUAAAAH ./apache/20180508/20180508-1513/20180508-151357-WvIFBQoeANwAAMwnE@4AAAAA ./apache/20180508/20180508-1513/20180508-151357-WvIFBQoeANwAAMwoFD8AAAAB ./apache/20180508/20180508-1516/20180508-151608-WvIFiAoeANwAAMz1FSwAAAAA ./apache/20180508/20180508-1516/20180508-151609-WvIFiQoeANwAAMz2FYIAAAAB ./apache/20180508/20180508-1516/20180508-151611-WvIFiwoeANwAAMz3FeEAAAAC ./apache/20180508/20180508-1516/20180508-151611-WvIFiwoeANwAAMz4Fj4AAAAD ./apache/20180508/20180508-2112/20180508-211205-WvJY9QoeANwAAM1MFCoAAAAA ### works with echo echo "./apache/20180508/20180508-1428/20180508-142802-WvH6QgoeANwAAMwsFZ4AAAAF" | sed 's/[^\/]*/\./;s/].*$//g' | awk '{print $0}' | xargs cat ### works with cat cat /var/log/httpd/modsec_audit.log | sed 's/[^\/]*/\./;s/].*$//g' | awk '{print $0}' | xargs cat ### does not work with tail... tail -f /var/log/httpd/modsec_audit.log | sed 's/[^\/]*/\./;s/].*$//g' | awk '{print $0}' | xargs cat I assume the tailing does not work because the script never terminates and sed is still caching the results until termination of the script. ***Is there a way to make this work, continuously?***

I'm encountering a strange behaviour with tail -f, when tailing a symlink on an Ubuntu machine. Apparently, tail uses the default update interval of 1 second. If tailing the original file, it seems to update immediately. lrwxrwxrwx 1 root root 49 Sep 9 20:28 firewall_log-01.log -> /var/data/log/firewall_log-01.log Doing $ tail -f firewall_log-01.log "pauses" every second (roughly, haven't measured exactly), which fits the defaults for tail: -s, --sleep-interval=N with -f, sleep for approximately N seconds (default 1.0) When doing $ tail -f /var/data/log/firewall_log-01.log the output gets updated immediately, apparently not concerned with the default setting for interval. When I run tail -s 0.01: $ tail -f -s 0.01 firewall_log-01.log I basically have what I need, but I'm still confused by why this even happens. I can do a workaround by adding an alias to ~/.bashrc, but this seems like a kind of dirty workaround. Is there any logical reason for tail to behave like this?

I have a log of 55GB in size. I tried:

cat logfile.log | tail

But this approach takes a lot of time. Is there any way to read huge files faster or any other approach?

when tailing multiple files at once as shown below, is any there any way to show the file name at the start of each line? tail -f one.log two.log current output ==> one.log two.log <== contents of one.log here... contents of two.log here.. Looking for something like one.log: contents of one.log here... one.log: contents of one.log here... two.log: contents of two.log here... two.log: contents of two.log here...

Problem: Need to run several process, especially it will be “tail -f log.log >> wrile_log.log” proces to collect log. Commands can be run at the different time for same log file. This is not problem, but.. How do I kill one of them process and not interrupted another?

So, I am working with tail -F ( or tail --follow=filename). Now it works as advertised and when a rollover occurs it will move to the new file. This is great and helps me keep track of my logs. The issue is I want to know when tail moves from old file to the new one. **The situation is as follows:** I have a huge log file that takes 15 mins to process. Now lets say at minute 5, a rollover occurs. Tail has the file description open, and uses it to complete tailing process and then moves to the new one. Now I keep a history of which file and which line I have last logged. I do this by increasing the number by the lines processes (its multiprocess program, but what else can I do?). The issue comes that after the rollover, the new file is starting from line 0, but my line_number has already been increase to 5 million. So, for this new file which has say 100 logs, I'll store the line number as 5,000,100. I used watchdog to find when the file rolls over to reset the line number to 0, but in case the rollover happens during the initial run, like say at 5minute mark of the 15 min run, then I still end up with a 3mil+ number. Since line_number is used to continue from where I last left in case the program died accidentally, this can mean loss of data. Just to note: I am running this command from python (CPython)

I need to extract information from a log file that is deleted and recreated every time a program runs. After detecting that the file exists (again), I would like to tail it for a certain regexp. The regexp will be matched a few times, but the result is always the same and I want to print it just once and after that go back to monitoring when the file is re-created. I looked at ways of detecting file creation. One way would be via inotifywait , but that requires installing a separate package. Perhaps a simpler way is to take advantage that tail prints to stderr when a file that is being tailed is deleted and created: tail: '/path/to/debug.log' has become inaccessible: No such file or directory tail: '/path/to/debug.log' has appeared; following new file So I applied this solution which is working: debug_file="/path/to/debug.log" while true; do # Monitor the log file until the 'new file' message appears ( tail -F $debug_file 2>&1 & ) | grep -q "has appeared; following new file" # After the new file message appears, switch to monitoring for the regexp tail -F "$debug_file" | while read -r line; do id=$(echo "$line" | sed -n 's/.* etc \([0-9]\+\),.*/\1/p') if [ -n "$id" ]; then echo "ID: $id" break # Exit the inner loop after the first match fi done done But I don't like that this solution starts 2 different tail processes. Is there a way to achieve the same result, but using just 1 tail process? And then switch 'modes', start by looking for file creation, then look for the regexp and once that is found go back to 'standby' mode waiting for the log file to be deleted and created again. Is inotifywait a more elegant solution? Ideally I would like a solution I could port easily to Windows CMD.

I have to report some logs from some of the servers in our infrastructure which I have limited permissions on. Every time I copy 1000 lines of logs and paste it into Slack, I can't do scp from the server to my local computer. I can only scp from my local and grab a file and paste it in a directory on my laptop. I cannot install any packages on this server as I have no permission like that. The command I'm using is this: tail -n 1000 ./mylogfile.log My process is this: 1. SSH into server 2. Run that tail command 2. Copy 1000 lines of code 3. Paste and report it in Slack What I want to achieve: 1. Run that command 2. Somehow get that log file on my local system as a file or in my clipboard (preferred), which is hard because of limitations I have My limitations are: 1. I can only see this server through VPN, the server can't see me, and I can't do SCP from the server. 2. I can't install any new packages In summary, I want to get that 1000 lines of log through an easy clean way without copying despite the limitations I have. Thank you everyone.