Sample Header Ad - 728x90

OOM Killer triggered when available memory is high

0 votes
2 answers
6203 views
I have been getting random kswapd0, and OOM killers even though available RAM -100MB. Already gone through many other similar issues, but I could not get why OOM killer triggered in my case. Hoping someone with knowledge can share some insight and set the direction for me to look into. EDIT: From top, I get this output when OOM killer triggered. Also, I am wondering why kswap triggered though ~100MB available? Our application needs only ~90 max, and already this has ~50MB allocated. So it was trying only ~40MB when this happened. top - 09:19:06 up 23:57, 0 users, load average: 4.50, 2.61, 1.87 Tasks: 101 total, 2 running, 99 sleeping, 0 stopped, 0 zombie %Cpu(s): 7.1 us, 62.5 sy, 0.0 ni, 0.0 id, 25.9 wa, 0.0 hi, 4.5 si, 0.0 st KiB Mem : 507008 total, 99320 free, 355096 used, 52592 buff/cache KiB Swap: 0 total, 0 free, 0 used. 102308 avail Mem top - 09:19:09 up 23:57, 0 users, load average: 4.50, 2.61, 1.87 Tasks: 100 total, 1 running, 98 sleeping, 0 stopped, 1 zombie %Cpu(s): 35.8 us, 45.4 sy, 0.0 ni, 0.0 id, 17.4 wa, 0.0 hi, 1.4 si, 0.0 st KiB Mem : 507008 total, 162280 free, 288952 used, 55776 buff/cache KiB Swap: 0 total, 0 free, 0 used. 168376 avail Mem Following is the back trace I have. 2022-07-29T09:19:09.117931Z,BTL200072600123,3,0,kernel:,[86254.933997] Out of memory: Kill process 25402 (application) score 181 or sacrifice child 2022-07-29T09:19:09.117941Z,BTL200072600123,3,0,kernel:,[86254.934006] Killed process 25402 (application) total-vm:159852kB, anon-rss:75664kB, file-rss:16020kB 2022-07-29T09:19:09.095963Z,BTL200072600123,4,1,kernel:, [86254.932989] acquisition invoked oom-killer: gfp_mask=0x2084d0, order=0, oom_score_adj=0 2022-07-29T09:19:09.096076Z,BTL200072600123,4,1,kernel:, [86254.933012] CPU: 0 PID: 17939 Comm: acquisition Tainted: G O 4.1.46 #5 2022-07-29T09:19:09.096142Z,BTL200072600123,4,1,kernel:, [86254.933019] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree) 2022-07-29T09:19:09.096206Z,BTL200072600123,4,1,kernel:, [86254.933025] Backtrace: 2022-07-29T09:19:09.096270Z,BTL200072600123,4,1,kernel:, [86254.933054] [] (dump_backtrace) from [] (show_stack+0x18/0x1c) 2022-07-29T09:19:09.096334Z,BTL200072600123,4,1,kernel:, [86254.933060] r7:00000000 r6:80a83c70 r5:600f0113 r4:00000000 202 2022-07-29T09:19:09.098354Z,BTL200072600123,4,1,kernel:, [86254.933467] Mem-Info: 2022-07-29T09:19:09.098411Z,BTL200072600123,4,1,kernel:, [86254.933485] active_anon:50599 inactive_anon:6859 isolated_anon:0 2022-07-29T09:19:09.098472Z,BTL200072600123,4,1,kernel:, [86254.933485] active_file:136 inactive_file:159 isolated_file:0 2022-07-29T09:19:09.098530Z,BTL200072600123,4,1,kernel:, [86254.933485] unevictable:16 dirty:0 writeback:0 unstable:0 2022-07-29T09:19:09.098589Z,BTL200072600123,4,1,kernel:, [86254.933485] slab_reclaimable:1089 slab_unreclaimable:2343 2022-07-29T09:19:09.098648Z,BTL200072600123,4,1,kernel:, [86254.933485] mapped:5971 shmem:8154 pagetables:534 bounce:0 2022-07-29T09:19:09.098704Z,BTL200072600123,4,1,kernel:, [86254.933485] free:23627 free_pcp:0 free_cma:23127 2022-07-29T09:19:09.098765Z,BTL200072600123,4,1,kernel:, [86254.933518] Normal free:94380kB min:1972kB low:2464kB high:2956kB active_anon:201792kB inactive_anon:27364kB active_file:476kB inactive_file:560kB unevictable:64kB isolated(anon):0kB isolated(file):0kB present:522240kB managed:505984kB mlocked:64kB dirty:0kB writeback:0kB mapped:23728kB shmem:32540kB slab_reclaimable:4356kB slab_unreclaimable:9372kB kernel_stack:1032kB pagetables:2136kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:92508kB writeback_tmp:0kB pages_scanned:6648 all_unreclaimable? yes 2022-07-29T09:19:09.098829Z,BTL200072600123,4,1,kernel:, [86254.933523] lowmem_reserve[]: 0 8 8 2022-07-29T09:19:09.098890Z,BTL200072600123,4,1,kernel:, [86254.933549] HighMem free:128kB min:128kB low:128kB high:132kB active_anon:604kB inactive_anon:72kB active_file:68kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:1024kB managed:1024kB mlocked:0kB dirty:0kB writeback:0kB mapped:156kB shmem:76kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no 2022-07-29T09:19:09.098950Z,BTL200072600123,4,1,kernel:, [86254.933555] lowmem_reserve[]: 0 0 0 2022-07-29T09:19:09.099011Z,BTL200072600123,4,1,kernel:, [86254.933564] Normal: 268*4kB (UEMRC) 128*8kB (UERC) 80*16kB (UERC) 8*32kB (RC) 0*64kB 1*128kB (C) 0*256kB 1*512kB (C) 0*1024kB 4*2048kB (C) 20*4096kB (C) = 94384kB 2022-07-29T09:19:09.099068Z,BTL200072600123,4,1,kernel:, [86254.933608] HighMem: 32*4kB (U) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 128kB 2022-07-29T09:19:09.099126Z,BTL200072600123,4,1,kernel:, [86254.933638] 8449 total pagecache pages 2022-07-29T09:19:09.099183Z,BTL200072600123,4,1,kernel:, [86254.933646] 0 pages in swap cache 2022-07-29T09:19:09.099240Z,BTL200072600123,4,1,kernel:, [86254.933652] Swap cache stats: add 0, delete 0, find 0/0 2022-07-29T09:19:09.099297Z,BTL200072600123,4,1,kernel:, [86254.933656] Free swap = 0kB 2022-07-29T09:19:09.099353Z,BTL200072600123,4,1,kernel:, [86254.933661] Total swap = 0kB 2022-07-29T09:19:09.099408Z,BTL200072600123,4,1,kernel:, [86254.933665] 130816 pages RAM 2022-07-29T09:19:09.099464Z,BTL200072600123,4,1,kernel:, [86254.933670] 256 pages HighMem/MovableOnly 2022-07-29T09:19:09.099521Z,BTL200072600123,4,1,kernel:, [86254.933675] 4294905824 pages reserved 2022-07-29T09:19:09.099578Z,BTL200072600123,4,1,kernel:, [86254.933680] 65536 pages cma reserved
Asked by Dinesh Kumar Govinda (1 rep)
Aug 15, 2022, 07:23 PM
Last activity: Aug 16, 2022, 07:48 AM