Sample Header Ad - 728x90

Is there a program that concatenates non-seekable streams (size not known in advance) and can separate them again?

1 vote
2 answers
99 views
I'm trying to concatenate multiple input files/streams into one stream (using the imaginary command stream-cat), pipe that stream into ssh and on the remote host separate it back into individual files/streams (stream-sep), like in this example, which is for demonstration purposes only:
stream-cat ( zfs receive tank/vm@snapshot ) somefile.txt"
Explanation of the example: zfs send outputs a large stream of data whose size isn't known in advance (that's why tar can't handle it). That data stream is concatenated with the content of a regular file somefile.txt. The resulting stream is piped into ssh, where it is separated again. The first stream gets piped into zfs receive, while the second is written to a regular file. Such a program should be straightforward to implement by reading non-seekable streams in chunks and always writing the chunk size followed by the data, until the end of the stream is reached. The overhead would be minimal. Does such a program exist already?
Asked by David Scherfgen (143 rep)
Jun 9, 2023, 10:56 PM
Last activity: Jun 10, 2023, 01:55 PM