Sample Header Ad - 728x90

Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?

3 votes
1 answer
18230 views
I'm excited to learn that xz now supports multithreading: xz --threads=0 But now I want to utilise this as much as possible. For example, to recompress gzips as xz: gzip -d -k -c myfile.gz | pv | xz -z --threads=0 - > myfile.xz This results in my processor being more highly used (~260% CPU to xz, yay!). However: - I realise that gzip is not (yet) multithreading, - I think that either pv or the pipes may be restricting the number of (IO?) threads. Is this true and, if so, is there a way to make this more efficient (other than to remove pv)?
Asked by tudor -Reinstate Monica- (545 rep)
Oct 23, 2019, 11:25 PM
Last activity: Sep 19, 2022, 02:17 PM