Sample Header Ad - 728x90

Uncompressed .lzo files in parallel in both the folders simultaneously and then delete the original .lzo files

1 vote
2 answers
414 views
So I have .lzo files in /test01/primary folder which I need to uncompress and then delete all the .lzo files. Same thing I need to do in /test02/secondary folder as well. I will have around 150 .lzo files in both folders so total around 300 .lzo files. From a command line I was running like this to uncomressed one file lzop -d file_name.lzo. What is the fastest way to uncompressed all .lzo files and then delete all .lzo files from both folders simultaneously. Below is the code I have: #!/bin/bash set -e export PRIMARY=/test01/primary export SECONDARY=/test02/secondary parallel lzop -dU -- ::: {"$PRIMARY","$SECONDARY"}/*.lzo I want to uncompress and delete .lzo files parallelly in both PRIMARY and SECONDARY folder simultaneously. With my above code, it does in PRIMARY first and then in SECONDARY folder. How can I achieve parallellism both in PRIMARY and SECONDARY simultaneously? Also does it uncompress all the files and then delete later on or uncompress one file and then delete that file and then move to next one? I tried with this but it doesn't work. It just works on first 40 files and after that it doesn't work at all. #!/bin/bash set -e export PRIMARY=/test01/primary export SECONDARY=/test02/secondary parallel -j 40 lzop -dU -- ::: "$PRIMARY"/*.lzo & parallel -j 40 lzop -dU -- ::: "$SECONDARY"/*.lzo & wait
Asked by user1950349 (841 rep)
Oct 9, 2015, 11:34 PM
Last activity: May 12, 2025, 08:12 AM