Sample Header Ad - 728x90

bash pipe loses data when command crashed

6 votes
1 answer
772 views
Consider this simple program (abort.py) that writes some text to stdout and stderr and then crashed (abort()).
import os
import sys

print("end stdout")
print("end stderr", file=sys.stderr)
os.abort()
When I run it in terminal, both output to stdout and stderr are correctly produced.
$ python3 abort.py 
end stdout
end stderr
Aborted (core dumped)
However, if I were to redirect stdout and/or stderr to another program (perhaps for logging purpose), it doesn't work any more.
$ python3 abort.py | cat
end stderr

$ python3 abort.py |& cat

$ python3 abort.py | tee -a logs
end stderr

$ python3 abort.py |& tee -a logs

# file logs is unchanged
In fact, if the program (abort.py) produce a lot of texts to stdout and stderr, **only the last section of them is lost** from the perspective of the receiving program on the other end of the pipe. I also tried running the program in a bash script and run that script, but the result is the same. Why is this happening and how should I fix it? ### Background The example above is obviously contrived. The real world scenario is that I was debugging a large program that occasionally crash (segfault or aborted) after running for days. It produces a lot of logs to both stdout and stderr, but the most important piece of information regarding its crash (such as stack trace) is printed in the end right before the crash happens. I have a logging pipeline setup in a bash script (involves some tee and gzip for example), but I found that last section of the log saved is always missing, which is frustrating.
Asked by lewisxy (171 rep)
Dec 3, 2024, 08:48 AM
Last activity: Dec 3, 2024, 03:48 PM