xz: OOM when compressing 1TB .tar

1 vote

1 answer

968 views

linux-kernel out-of-memory multithreading xz

I'm trying to compress a large archive with multi-threading enabled, however, my system keeps freezing up and runs out of memory.

OS: Manjaro 21.1.0 Pahvo    
Kernel: x86_64 Linux 5.13.1-3-MANJARO
Shell: bash 5.1.9
RAM: 16GB

|swapon|
NAME      TYPE   SIZE   USED   PRIO
/swapfile file   32G    0B     -2

I've tried this with a /swapfile 2x the amount of RAM I have (32GB) but the system would always freeze once >90% of total RAM has been used, and would seem to not make use of the /swapfile.

|xz --info-memory|
Total amount of physical memory (RAM) : 15910 MiB
Memory usage limit for compression: Disabled
Memory usage limit for decompression: Disabled

I'm new to using xz so please bear with me, but is there a way to globally enable the memory usage limiter and for the Total amount of physical memory (RAM) to take into account the space made available by /swapfile?

Asked by JKleinne (113 rep)

Jul 23, 2021, 10:20 PM
Last activity: Jul 23, 2021, 10:55 PM

xz: OOM when compressing 1TB .tar

Related Questions