Can I make a pipe save data to disk instead of blocking until the reader can continue?
4
votes
1
answer
594
views
Are there any established tools where the pipe spills to disk rather than blocking the upstream process?
As an example, in a traditional pipeline A | B, we get the following behavior when B does not read from stdin:
- A emits output until stdout fills up
- Then A is blocked until B reads from stdin
I'd like the following behavior instead:
- A emits output until its stdout buffer fills up
- Further output from A is written to an on-disk cache file, so A is not blocked
- When B ingests data from stdin, new data from the on-disk cache file is read (FIFO) into the buffer
Is there any existing tool that accomplishes this?
Thank you!
Asked by Autodidactyle
(151 rep)
Nov 26, 2024, 04:53 PM
Last activity: Nov 27, 2024, 09:33 AM
Last activity: Nov 27, 2024, 09:33 AM