Is there a program that concatenates non-seekable streams (size not known in advance) and can separate them again?
1
vote
2
answers
99
views
I'm trying to concatenate multiple input files/streams into one stream (using the imaginary command
stream-cat
), pipe that stream into ssh
and on the remote host separate it back into individual files/streams (stream-sep
), like in this example, which is for demonstration purposes only:
stream-cat ( zfs receive tank/vm@snapshot ) somefile.txt"
Explanation of the example: zfs send
outputs a large stream of data whose size isn't known in advance (that's why tar
can't handle it). That data stream is concatenated with the content of a regular file somefile.txt
. The resulting stream is piped into ssh
, where it is separated again. The first stream gets piped into zfs receive
, while the second is written to a regular file.
Such a program should be straightforward to implement by reading non-seekable streams in chunks and always writing the chunk size followed by the data, until the end of the stream is reached. The overhead would be minimal.
Does such a program exist already?
Asked by David Scherfgen
(143 rep)
Jun 9, 2023, 10:56 PM
Last activity: Jun 10, 2023, 01:55 PM
Last activity: Jun 10, 2023, 01:55 PM