Sample Header Ad - 728x90

How to analyse cause of memory usage not shown by htop

0 votes
1 answer
196 views
**How can I best debug the following issue from here on?** I have a - python application - running in a **podman container** which has - plenty disk I/O due to a **numpy memmap** of two huge files, one input file which I read in chunks, process the chunk and write the result in the other mem mapped file. I noticed that running the application leads to - **increasing total RAM usage**, eventually leading to - OOM situations on my system. I tried finding the cause but - couldn't find references in my python code which would leak memory. According to sudo htop - the application itself **does not consume the memory** when accounting for resident memory size only, which it defines as text sections + data sections + stack usage. - Mem 11.0G / 14.6G - 18.7 MEM% (of ~16G RAM) later: - Mem 14.3 / 14.6G - 0.1 MEM% (of ~16G RAM) According to ps auwx: |USER|PID|%CPU|%MEM|VSZ|RSS|TTY|STAT|START|TIME|COMMAND |--|--|--|--|--|--|--|--|--|--|--|--|--|--|--| |user|9874|44.9|0.0|14651602572|8092|pts/0|DNsl+|10:40|275:13|/root/.local/share/virtualenvs/app-4PlAip0Q/bin/python main.py
# /proc/meminfo
MemTotal:       15258424 kB
MemFree:          143284 kB
MemAvailable:          0 kB
Buffers:             304 kB
Cached:            51496 kB
SwapCached:         1380 kB
Active:              812 kB
Inactive:           5800 kB
Active(anon):          0 kB
Inactive(anon):     1492 kB
Active(file):        812 kB
Inactive(file):     4308 kB
Unevictable:       46068 kB
Mlocked:               0 kB
SwapTotal:      67108860 kB
SwapFree:       66860540 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:               144 kB
Writeback:           124 kB
AnonPages:          1276 kB
Mapped:             2916 kB
Shmem:             46020 kB
KReclaimable:      40196 kB
Slab:            1482400 kB
SReclaimable:      40196 kB
SUnreclaim:      1442204 kB
KernelStack:        3984 kB
PageTables:     13424812 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    74737048 kB
Committed_AS:    1396560 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       54808 kB
VmallocChunk:          0 kB
Percpu:             2496 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       1
HugePages_Free:        1
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:            2048 kB
DirectMap4k:      464816 kB
DirectMap2M:    14129152 kB
DirectMap1G:     2097152 kB
**Screenshots** htop htop
Asked by Semnodime (397 rep)
May 24, 2024, 02:07 AM
Last activity: May 24, 2024, 02:25 AM