Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?
3
votes
1
answer
18230
views
I'm excited to learn that
xz
now supports multithreading:
xz --threads=0
But now I want to utilise this as much as possible. For example, to recompress gzips as xz:
gzip -d -k -c myfile.gz | pv | xz -z --threads=0 - > myfile.xz
This results in my processor being more highly used (~260% CPU to xz, yay!).
However:
- I realise that gzip
is not (yet) multithreading,
- I think that either pv
or the pipes may be restricting the number of (IO?) threads.
Is this true and, if so, is there a way to make this more efficient (other than to remove pv
)?
Asked by tudor -Reinstate Monica-
(545 rep)
Oct 23, 2019, 11:25 PM
Last activity: Sep 19, 2022, 02:17 PM
Last activity: Sep 19, 2022, 02:17 PM