有大量可用交换空间时调用 OOM

有大量可用交换空间时调用 OOM

我正在运行一个带有 i2p 和 tor 的 ARM 路由器 - Netgear R7000。当然,我为其添加了整整 512 MB 的 SWAP 以防止 OOM,我知道这可能会降低系统速度……但我仍然得到 OOM 杀手,从大量 SWAP 可用开始,并杀死 tor!更有趣的是,在杀死 tor 之后,系统似乎可以无限期地正常运行……但似乎仍然无法交换 tor。我甚至尝试关闭过度使用,但根本没有帮助。

参见下面的 dmesg 日志

resetbutton invoked oom-killer: gfp_mask=0x2000d0, order=0, oom_score_adj=0
CPU: 0 PID: 1500 Comm: resetbutton Tainted: P             3.10.79 #381
Backtrace:
[<c0015cb8>] (dump_backtrace+0x0/0x118) from [<c0015ec0>] (show_stack+0x18/0x1c)
 r6:c7a4cdc0 r5:00000000 r4:c6818000 r3:00000000
[<c0015ea8>] (show_stack+0x0/0x1c) from [<c012e5c0>] (dump_stack+0x24/0x28)
[<c012e59c>] (dump_stack+0x0/0x28) from [<c007c5cc>] (dump_header.isra.13+0x84/0x194)
[<c007c548>] (dump_header.isra.13+0x0/0x194) from [<c007c958>] (oom_kill_process+0x90/0x3e8)
[<c007c8c8>] (oom_kill_process+0x0/0x3e8) from [<c007d17c>] (out_of_memory+0x2c0/0x304)
[<c007cebc>] (out_of_memory+0x0/0x304) from [<c007fcbc>] (__alloc_pages_nodemask+0x5b0/0x694)
[<c007f70c>] (__alloc_pages_nodemask+0x0/0x694) from [<c00a2464>] (cache_alloc_refill+0x2c8/0x600)
[<c00a219c>] (cache_alloc_refill+0x0/0x600) from [<c00a214c>] (kmem_cache_alloc+0x88/0xd8)
[<c00a20c4>] (kmem_cache_alloc+0x0/0xd8) from [<c00afdd4>] (getname_flags+0x2c/0x104)
 r7:b6f5509c r6:00000000 r5:00000000 r4:00020000
[<c00afda8>] (getname_flags+0x0/0x104) from [<c00afec4>] (getname+0x18/0x1c)
[<c00afeac>] (getname+0x0/0x1c) from [<c00a5270>] (do_sys_open+0xbc/0x178)
[<c00a51b4>] (do_sys_open+0x0/0x178) from [<c00a5350>] (SyS_open+0x24/0x28)
 r9:c6818000 r8:c0008d24 r7:00000005 r6:00000000 r5:00000000
r4:00000000
[<c00a532c>] (SyS_open+0x0/0x28) from [<c0008ba0>] (ret_fast_syscall+0x0/0x30)
Mem-info:
Normal per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
HighMem per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
active_anon:10263 inactive_anon:12797 isolated_anon:0
 active_file:957 inactive_file:1335 isolated_file:0
 unevictable:151 dirty:0 writeback:0 unstable:0
 free:13787 slab_reclaimable:912 slab_unreclaimable:5734
 mapped:1171 shmem:0 pagetables:283 bounce:0
 free_cma:0
Normal free:26184kB min:32768kB low:40960kB high:49152kB active_anon:0kB inactive_anon:0kB active_file:96kB inactive_file:64kB unevictable:64kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:124196kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:3648kB slab_unreclaimable:22936kB kernel_stack:1192kB pagetables:1132kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:4440 all_unreclaimable? no
lowmem_reserve[]: 0 1024 1024
HighMem free:28656kB min:128kB low:8772kB high:17416kB active_anon:41052kB inactive_anon:51512kB active_file:3764kB inactive_file:5244kB unevictable:540kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:131072kB mlocked:0kB dirty:0kB writeback:0kB mapped:4684kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Normal: 102*4kB (UEM) 118*8kB (UEM) 360*16kB (UM) 158*32kB (UM) 28*64kB (UEM) 8*128kB (UM) 4*256kB (UM) 4*512kB (UM) 0*1024kB 0*2048kB 2*4096kB (R) = 26248kB
HighMem: 1177*4kB (UMR) 1620*8kB (UMR) 225*16kB (UMR) 30*32kB (UMR) 29*64kB (UMR) 8*128kB (UMR) 1*256kB (U) 2*512kB (U) 0*1024kB 1*2048kB (U) 0*4096kB = 28436kB
9347 total pagecache pages
6914 pages in swap cache
Swap cache stats: add 484419, delete 477505, find 187469/228822
Free swap  = 408788kB
Total swap = 524284kB
65536 pages of RAM
13904 free pages
1661 reserved pages
6646 slab pages
549590 pages shared
6923 pages swap cached
[ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[  622]     0   622      189        7       3       13             0 hotplug2
[  626]     0   626      199       30       3        6             0 mstpd
[  971]     0   971      217        9       3        8             0 dropbear
[  987]     0   987      297      105       3       30             0 dnsmasq
[ 1112]     0  1112      315       29       4       30             0 ttraff
[ 1195]     0  1195      209       25       3        6             0 dhcp6c
[ 1288]     0  1288      755        3       4       83             0 httpd
[ 1330]     0  1330      315       26       3       18             0 process_monitor
[ 1334]     0  1334      361       44       3       38             0 upnp
[ 1337]     0  1337      334       50       4        9             0 wland
[ 1338]     0  1338      190       42       3        3             0 igmprt
[ 1339]     0  1339      291       10       3        6             0 udhcpc
[ 1449]     0  1449      182       27       3        3             0 cron
[ 1457]     0  1457      370       38       3       32             0 nas
[ 1460]     0  1460      370       36       4       32             0 nas
[ 1495]     0  1495      291       11       3        6             0 syslogd
[ 1497]     0  1497      291       15       3        6             0 klogd
[ 1500]     0  1500      334       32       3       12             0 resetbutton
[ 1684] 65534  1684     1008       18       5      499             0 tinyproxy
[ 1690] 65534  1690     1021        9       5      508             0 tinyproxy
[ 1691] 65534  1691     1021        9       5      508             0 tinyproxy
[ 1692] 65534  1692     1021        9       5      508             0 tinyproxy
[ 1693] 65534  1693     1008        6       5      502             0 tinyproxy
[ 1694] 65534  1694     1008        6       5      502             0 tinyproxy
[ 1695] 65534  1695     1008        6       5      502             0 tinyproxy
[ 1696] 65534  1696     1008        6       5      502             0 tinyproxy
[ 1697] 65534  1697     1008        6       5      502             0 tinyproxy
[ 1698] 65534  1698     1008        6       5      502             0 tinyproxy
[ 1699] 65534  1699     1008        6       5      502             0 tinyproxy
[ 1711]   106  1711    36556    12756      73    10236             0 tor
[ 1730]    13  1730      580        9       4       38             0 polipo
[ 1806]  1000  1806     2828       41       5      124             0 wrapper
[ 1808]  1000  1808    59927     7979      69    13797             0 java
[ 1921]     0  1921      241       18       3       31             0 dropbear
[ 1923]     0  1923      293        9       3        9             0 sh
[ 3705]     0  3705      804        5       4       86             0 bash
[ 5859]     0  5859      320       61       3       12             0 radvd
[ 5860]     0  5860      320       18       3       18             0 radvd
Out of memory: Kill process 1711 (tor) score 118 or sacrifice child
Killed process 1711 (tor) total-vm:146224kB, anon-rss:49036kB, file-rss:1988kB
tor: page allocation failure: order:0, mode:0x20058
CPU: 1 PID: 1711 Comm: tor Tainted: P             3.10.79 #381
Backtrace:
[<c0015cb8>] (dump_backtrace+0x0/0x118) from [<c0015ec0>] (show_stack+0x18/0x1c)
 r6:00000000 r5:00000000 r4:00020058 r3:00000000
[<c0015ea8>] (show_stack+0x0/0x1c) from [<c012e5c0>] (dump_stack+0x24/0x28)
[<c012e59c>] (dump_stack+0x0/0x28) from [<c007def4>] (warn_alloc_failed+0x100/0x110)
[<c007ddf4>] (warn_alloc_failed+0x0/0x110) from [<c007faf0>] (__alloc_pages_nodemask+0x3e4/0x694)
 r3:00040001 r2:00000000
 r7:c0405180 r6:00000001 r5:00000000 r4:00020058
[<c007f70c>] (__alloc_pages_nodemask+0x0/0x694) from [<c007a274>] (find_or_create_page+0x48/0x94)
[<c007a22c>] (find_or_create_page+0x0/0x94) from [<c00cfc08>] (__getblk+0x16c/0x308)
 r9:00000000 r8:0223764c r7:c7418110 r6:00000001 r5:00001000
r4:00000000
[<c00cfa9c>] (__getblk+0x0/0x308) from [<c00d0668>] (__bread+0x1c/0xd8)
[<c00d064c>] (__bread+0x0/0xd8) from [<bf5e96c8>] (_8+0x208/0x4a0 [ext2])
 r4:c6bb5b58
[<bf5e963c>] (_8+0x17c/0x4a0 [ext2]) from [<bf5ea2e0>] (_35+0x78/0x884 [ext2])
[<bf5ea268>] (_35+0x0/0x884 [ext2]) from [<c00d6da8>] (do_mpage_readpage+0x268/0x6d4)
[<c00d6b40>] (do_mpage_readpage+0x0/0x6d4) from [<c00d72d4>] (mpage_readpages+0xc0/0x104)
[<c00d7214>] (mpage_readpages+0x0/0x104) from [<bf5e99ac>] (_61+0x24/0x28 [ext2])
[<bf5e9988>] (_61+0x0/0x28 [ext2]) from [<c0082f08>] (__do_page_cache_readahead+0x170/0x1f8)
[<c0082d98>] (__do_page_cache_readahead+0x0/0x1f8) from [<c0083428>] (ra_submit+0x30/0x38)
[<c00833f8>] (ra_submit+0x0/0x38) from [<c007bb4c>] (filemap_fault+0x1bc/0x434)
[<c007b990>] (filemap_fault+0x0/0x434) from [<c008e5a4>] (__do_fault+0xac/0x490)
[<c008e4f8>] (__do_fault+0x0/0x490) from [<c0091284>] (handle_pte_fault+0x78/0x6d4)
[<c009120c>] (handle_pte_fault+0x0/0x6d4) from [<c0091968>] (__handle_mm_fault+0x88/0x90)
[<c00918e0>] (__handle_mm_fault+0x0/0x90) from [<c00920e0>] (handle_mm_fault+0x28/0x2c)
[<c00920b8>] (handle_mm_fault+0x0/0x2c) from [<c001948c>] (do_page_fault+0x118/0x288)
[<c0019374>] (do_page_fault+0x0/0x288) from [<c0008580>] (do_DataAbort+0x40/0x9c)
[<c0008540>] (do_DataAbort+0x0/0x9c) from [<c0008998>] (__dabt_usr+0x38/0x40)
Exception stack(0xc6bb5fb0 to 0xc6bb5ff8)
5fa0:                                     b6fe43d8 b6cfd864 00000005 00000053
5fc0: 00000001 b6fe43dd b6d0a358 b6cfd864 b866141c 00000000 b6fe43d8 00000000
5fe0: b6d0b658 be8798f0 b6cf2330 b6a4c388 20000010 ffffffff
 r8:b866141c r7:b6cfd864 r6:ffffffff r5:20000010 r4:b6a4c388
Mem-info:
Normal per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
HighMem per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
active_anon:10263 inactive_anon:13171 isolated_anon:0
 active_file:971 inactive_file:1322 isolated_file:0
 unevictable:151 dirty:1 writeback:0 unstable:0
 free:13621 slab_reclaimable:912 slab_unreclaimable:5739
 mapped:1172 shmem:0 pagetables:283 bounce:0
 free_cma:0
Normal free:27136kB min:32768kB low:40960kB high:49152kB active_anon:0kB inactive_anon:0kB active_file:40kB inactive_file:120kB unevictable:64kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:124196kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:3648kB slab_unreclaimable:22956kB kernel_stack:1192kB pagetables:1132kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:40 all_unreclaimable? no
lowmem_reserve[]: 0 1024 1024
HighMem free:27380kB min:128kB low:8772kB high:17416kB active_anon:41052kB inactive_anon:52684kB active_file:3764kB inactive_file:5248kB unevictable:540kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:131072kB mlocked:0kB dirty:4kB writeback:0kB mapped:4688kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Normal: 96*4kB (UEM) 120*8kB (UEM) 360*16kB (UM) 182*32kB (UM) 32*64kB (UEM) 8*128kB (UM) 4*256kB (UM) 4*512kB (UM) 0*1024kB 0*2048kB 2*4096kB (R) = 27264kB
HighMem: 918*4kB (UMR) 1620*8kB (UMR) 225*16kB (UMR) 30*32kB (UMR) 29*64kB (UMR) 8*128kB (UMR) 1*256kB (U) 2*512kB (U) 0*1024kB 1*2048kB (U) 0*4096kB = 27400kB
9533 total pagecache pages
7099 pages in swap cache
Swap cache stats: add 484625, delete 477526, find 187576/228974
Free swap  = 408868kB
Total swap = 524284kB
65536 pages of RAM
13939 free pages
1661 reserved pages
6652 slab pages
549523 pages shared
7099 pages swap cached
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets

sysctl vm相关设置:

vm.admin_reserve_kbytes = 7914
vm.block_dump = 0
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 500
vm.drop_caches = 0
vm.highmem_is_dirtyable = 0
vm.laptop_mode = 0
vm.legacy_va_layout = 0
vm.lowmem_reserve_ratio = 32    32
vm.max_map_count = 65530
vm.min_free_kbytes = 32768
vm.mmap_min_addr = 4096
vm.nr_pdflush_threads = 0
vm.oom_dump_tasks = 1
vm.oom_kill_allocating_task = 0
vm.overcommit_memory = 2
vm.overcommit_ratio = 50
vm.page-cluster = 3
vm.panic_on_oom = 0
vm.percpu_pagelist_fraction = 0
vm.scan_unevictable_pages = 0
vm.stat_interval = 1
vm.swappiness = 100
vm.user_reserve_kbytes = 7914
vm.vfs_cache_pressure = 100

我做错了什么(除了使用 SOHO 路由器进行 tor,哈哈)?

PS:使用 DD-WRT 固件

编辑:另一个 OOM 杀手日志 - 这次没有关于 resetbutton 的内容:

kthreadd invoked oom-killer: gfp_mask=0x3000d0, order=1, oom_score_adj=0
CPU: 0 PID: 2 Comm: kthreadd Tainted: P             3.10.79 #381
Backtrace:
[<c0015cb8>] (dump_backtrace+0x0/0x118) from [<c0015ec0>] (show_stack+0x18/0x1c)
 r6:c782fb80 r5:00000000 r4:c7834000 r3:00000000
[<c0015ea8>] (show_stack+0x0/0x1c) from [<c012e5c0>] (dump_stack+0x24/0x28)
[<c012e59c>] (dump_stack+0x0/0x28) from [<c007c5cc>] (dump_header.isra.13+0x84/0x194)
[<c007c548>] (dump_header.isra.13+0x0/0x194) from [<c007c958>] (oom_kill_process+0x90/0x3e8)
[<c007c8c8>] (oom_kill_process+0x0/0x3e8) from [<c007d17c>] (out_of_memory+0x2c0/0x304)
[<c007cebc>] (out_of_memory+0x0/0x304) from [<c007fcbc>] (__alloc_pages_nodemask+0x5b0/0x694)
[<c007f70c>] (__alloc_pages_nodemask+0x0/0x694) from [<c002bf70>] (copy_process.part.68+0xdc/0xe30)
[<c002be94>] (copy_process.part.68+0x0/0xe30) from [<c002cdd8>] (do_fork+0xe0/0x2bc)
[<c002ccf8>] (do_fork+0x0/0x2bc) from [<c002cfe4>] (kernel_thread+0x30/0x38)
 r9:00000001 r8:00000000 r7:c7834000 r6:c0454014 r5:c04096a8
r4:c1109e88
[<c002cfb4>] (kernel_thread+0x0/0x38) from [<c004be50>] (kthreadd+0xe4/0x12c)
[<c004bd6c>] (kthreadd+0x0/0x12c) from [<c0008c38>] (ret_from_fork+0x14/0x3c)
Mem-info:
Normal per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
HighMem per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
active_anon:7196 inactive_anon:7223 isolated_anon:0
 active_file:1054 inactive_file:1422 isolated_file:0
 unevictable:199 dirty:3 writeback:0 unstable:0
 free:22911 slab_reclaimable:969 slab_unreclaimable:6549
 mapped:685 shmem:0 pagetables:210 bounce:0
 free_cma:0
Normal free:29372kB min:32768kB low:40960kB high:49152kB active_anon:0kB inactive_anon:0kB active_file:44kB inactive_file:128kB unevictable:44kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:124196kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:3876kB slab_unreclaimable:26196kB kernel_stack:576kB pagetables:840kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:1064 all_unreclaimable? no
lowmem_reserve[]: 0 1024 1024
HighMem free:62272kB min:128kB low:8772kB high:17416kB active_anon:28784kB inactive_anon:28892kB active_file:4068kB inactive_file:5676kB unevictable:752kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:131072kB mlocked:0kB dirty:12kB writeback:0kB mapped:2740kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Normal: 465*4kB (UEM) 285*8kB (UEM) 219*16kB (UEM) 92*32kB (UEM) 51*64kB (UEM) 25*128kB (UEM) 12*256kB (UEM) 2*512kB (U) 0*1024kB 0*2048kB 2*4096kB (R) = 29340kB
HighMem: 5997*4kB (UM) 2743*8kB (UM) 478*16kB (UM) 27*32kB (UM) 2*64kB (M) 2*128kB (U) 1*256kB (U) 2*512kB (U) 0*1024kB 1*2048kB (U) 1*4096kB (R) = 62252kB
5489 total pagecache pages
2805 pages in swap cache
Swap cache stats: add 1237151, delete 1234346, find 539424/688368
Free swap  = 484712kB
Total swap = 524284kB
65536 pages of RAM
23095 free pages
1661 reserved pages
7517 slab pages
557492 pages shared
2805 pages swap cached
[ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[  622]     0   622      189        1       3       12             0 hotplug2
[  626]     0   626      199       31       3        6             0 mstpd
[  971]     0   971      217        1       3        7             0 dropbear
[  987]     0   987      297      105       3       30             0 dnsmasq
[ 1112]     0  1112      315       61       4       10             0 ttraff
[ 1195]     0  1195      209       40       3        2             0 dhcp6c
[ 1288]     0  1288      755        1       4       82             0 httpd
[ 1330]     0  1330      315       38       3       17             0 process_monitor
[ 1334]     0  1334      366       89       3       11             0 upnp
[ 1337]     0  1337      334       50       4        9             0 wland
[ 1338]     0  1338      190       45       3        4             0 igmprt
[ 1339]     0  1339      291       30       3        5             0 udhcpc
[ 1449]     0  1449      182       32       3        2             0 cron
[ 1457]     0  1457      370       36       3       32             0 nas
[ 1460]     0  1460      370       34       4       32             0 nas
[ 1495]     0  1495      291       20       3        5             0 syslogd
[ 1497]     0  1497      291       37       3        2             0 klogd
[ 1500]     0  1500      334       32       3       12             0 resetbutton
[ 1684] 65534  1684     1008       15       5      499             0 tinyproxy
[ 1690] 65534  1690     1021        5       5      507             0 tinyproxy
[ 1691] 65534  1691     1021        5       5      507             0 tinyproxy
[ 1692] 65534  1692     1021        5       5      507             0 tinyproxy
[ 1693] 65534  1693     1008        4       5      501             0 tinyproxy
[ 1694] 65534  1694     1008        4       5      501             0 tinyproxy
[ 1695] 65534  1695     1008        4       5      501             0 tinyproxy
[ 1696] 65534  1696     1008        4       5      501             0 tinyproxy
[ 1697] 65534  1697     1008        4       5      501             0 tinyproxy
[ 1698] 65534  1698     1008        4       5      501             0 tinyproxy
[ 1699] 65534  1699     1008        4       5      501             0 tinyproxy
[ 1730]    13  1730      580       10       4       37             0 polipo
[ 1806]  1000  1806     2828       78       5      116             0 wrapper
[ 1921]     0  1921      241       51       3       19             0 dropbear
[ 1923]     0  1923      293        1       3        8             0 sh
[16658]   106 16643    33786    12722      65     7647             0 tor
[16661]     0 16661      804        4       4       85             0 bash
[17143]     0 17143      790      167       4       26             0 top
[20851]     0 20851      320       64       3       12             0 radvd
[20853]     0 20853      320       33       3       17             0 radvd
Out of memory: Kill process 16658 (tor) score 104 or sacrifice child
Killed process 16658 (tor) total-vm:135144kB, anon-rss:49088kB, file-rss:1800kB
tor: page allocation failure: order:0, mode:0x20058
CPU: 1 PID: 16658 Comm: tor Tainted: P             3.10.79 #381
Backtrace:
[<c0015cb8>] (dump_backtrace+0x0/0x118) from [<c0015ec0>] (show_stack+0x18/0x1c)
 r6:00000000 r5:00000000 r4:00020058 r3:00000000
[<c0015ea8>] (show_stack+0x0/0x1c) from [<c012e5c0>] (dump_stack+0x24/0x28)
[<c012e59c>] (dump_stack+0x0/0x28) from [<c007def4>] (warn_alloc_failed+0x100/0x110)
[<c007ddf4>] (warn_alloc_failed+0x0/0x110) from [<c007faf0>] (__alloc_pages_nodemask+0x3e4/0x694)
 r3:00040001 r2:00000000
 r7:c0405180 r6:c6488000 r5:00000000 r4:00020058
[<c007f70c>] (__alloc_pages_nodemask+0x0/0x694) from [<c007a274>] (find_or_create_page+0x48/0x94)
[<c007a22c>] (find_or_create_page+0x0/0x94) from [<c00cfc08>] (__getblk+0x16c/0x308)
 r9:00000000 r8:0220f844 r7:c7418110 r6:00000001 r5:00001000
r4:00000000
[<c00cfa9c>] (__getblk+0x0/0x308) from [<c00d0668>] (__bread+0x1c/0xd8)
[<c00d064c>] (__bread+0x0/0xd8) from [<bf5e96c8>] (_8+0x208/0x4a0 [ext2])
 r4:c6489b58
[<bf5e963c>] (_8+0x17c/0x4a0 [ext2]) from [<bf5ea2e0>] (_35+0x78/0x884 [ext2])
[<bf5ea268>] (_35+0x0/0x884 [ext2]) from [<c00d6da8>] (do_mpage_readpage+0x268/0x6d4)
[<c00d6b40>] (do_mpage_readpage+0x0/0x6d4) from [<c00d72d4>] (mpage_readpages+0xc0/0x104)
[<c00d7214>] (mpage_readpages+0x0/0x104) from [<bf5e99ac>] (_61+0x24/0x28 [ext2])
[<bf5e9988>] (_61+0x0/0x28 [ext2]) from [<c0082f08>] (__do_page_cache_readahead+0x170/0x1f8)
[<c0082d98>] (__do_page_cache_readahead+0x0/0x1f8) from [<c0083428>] (ra_submit+0x30/0x38)
[<c00833f8>] (ra_submit+0x0/0x38) from [<c007bb4c>] (filemap_fault+0x1bc/0x434)
[<c007b990>] (filemap_fault+0x0/0x434) from [<c008e5a4>] (__do_fault+0xac/0x490)
[<c008e4f8>] (__do_fault+0x0/0x490) from [<c0091284>] (handle_pte_fault+0x78/0x6d4)
[<c009120c>] (handle_pte_fault+0x0/0x6d4) from [<c0091968>] (__handle_mm_fault+0x88/0x90)
[<c00918e0>] (__handle_mm_fault+0x0/0x90) from [<c00920e0>] (handle_mm_fault+0x28/0x2c)
[<c00920b8>] (handle_mm_fault+0x0/0x2c) from [<c001948c>] (do_page_fault+0x118/0x288)
[<c0019374>] (do_page_fault+0x0/0x288) from [<c000861c>] (do_PrefetchAbort+0x40/0x9c)
[<c00085dc>] (do_PrefetchAbort+0x0/0x9c) from [<c0008b58>] (ret_from_exception+0x0/0x10)
Exception stack(0xc6489fb0 to 0xc6489ff8)
9fa0:                                     0000021c 00000000 6df1d800 6df1d800
9fc0: b470b880 0000000f da4afeed b6fe04bc 00000040 5eedf00d b6fdb8f4 b6fba134
9fe0: 00000000 b470b858 b6f0ead4 b6f0ead4 60000010 ffffffff
 r8:00000040 r7:b6fe04bc r6:ffffffff r5:60000010 r4:b6f0ead4
Mem-info:
Normal per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
HighMem per-cpu:
CPU    0: hi:   42, btch:   7 usd:   0
CPU    1: hi:   42, btch:   7 usd:   0
active_anon:7196 inactive_anon:7223 isolated_anon:0
 active_file:1028 inactive_file:1457 isolated_file:0
 unevictable:199 dirty:3 writeback:0 unstable:0
 free:22844 slab_reclaimable:969 slab_unreclaimable:6547
 mapped:685 shmem:0 pagetables:210 bounce:0
 free_cma:0
Normal free:29092kB min:32768kB low:40960kB high:49152kB active_anon:0kB inactive_anon:0kB active_file:44kB inactive_file:128kB unevictable:44kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:124196kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:3876kB slab_unreclaimable:26188kB kernel_stack:576kB pagetables:840kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 1024 1024
HighMem free:62252kB min:128kB low:8772kB high:17416kB active_anon:28784kB inactive_anon:28892kB active_file:4068kB inactive_file:5700kB unevictable:752kB isolated(anon):0kB isolated(file):0kB present:131072kB managed:131072kB mlocked:0kB dirty:12kB writeback:0kB mapped:2740kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Normal: 467*4kB (UEM) 285*8kB (UEM) 219*16kB (UEM) 93*32kB (UEM) 50*64kB (EM) 25*128kB (UEM) 11*256kB (UEM) 2*512kB (U) 0*1024kB 0*2048kB 2*4096kB (R) = 29060kB
HighMem: 5997*4kB (UM) 2743*8kB (UM) 478*16kB (UM) 27*32kB (UM) 2*64kB (M) 2*128kB (U) 1*256kB (U) 2*512kB (U) 0*1024kB 1*2048kB (U) 1*4096kB (R) = 62252kB
5489 total pagecache pages
2805 pages in swap cache
Swap cache stats: add 1237151, delete 1234346, find 539424/688368
Free swap  = 484712kB
Total swap = 524284kB
65536 pages of RAM
23025 free pages
1661 reserved pages
7515 slab pages
819780 pages shared
2805 pages swap cached
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets
TCP: too many orphaned sockets

再次,可用交换 = 484712kB - 大量未使用。为什么?

答案1

你的问题实际上就在这里:

Normal 
free:29372kB
min:32768kB 
low:40960kB
high:49152kB
active_anon:0kB 
inactive_anon:0kB 
active_file:44kB 
inactive_file:128kB 
unevictable:44kB 
isolated(anon):0kB 
isolated(file):0kB 
present:131072kB 
managed:124196kB 
mlocked:0kB 
dirty:0kB 
writeback:0kB 
mapped:0kB 
shmem:0kB 
slab_reclaimable:3876kB 
slab_unreclaimable:26196kB 
kernel_stack:576kB 
pagetables:840kB 
unstable:0kB 
bounce:0kB 
free_cma:0kB 
writeback_tmp:0kB 
pages_scanned:1064 
all_unreclaimable? no

路由器上的内存被分成多个区域,一个区域 - HighMem 包含 128MiB 内存,另一个区域 - Normal 包含 128MiB 内存。

这些区域在功能上彼此独立。内存管理器以独立的方式在每个区域中工作。

oom-killer 在正常区域中被调用。当您查看请求时,您实际上可以消除 128Mib 的内存。

现在来看看实际的异常情况:

free: 29372kB
min: 32768kB

min字段是内核对“当我低于此阈值时,我将不得不采取激烈措施来回收内存 - 例如 OOM-killer”的解释。什么是高度您的描述中关于此字段的不寻常之处在于,最小值被设置为多少。此值通常浮动在内存的 2% 左右,在本例中为整个区域的 25%。难怪您如此积极地进行 oom-kill!

当你查看问题输出时,问题的原因如下:

vm.min_free_kbytes = 32768

将此值更改为方式更低。或许1024这是一个更合理的数字。

您还会看到一些“孤立的 TCP 套接字”错误。您可能还希望减少和的值,net.ipv4.tcp_wmemnet.ipv4.tcp_rmem将允许您处理更多套接字(以延迟为代价)。

因此,回答您的问题,为什么要终止具有如此多可用交换的进程 - 答案是,vm.min_free_kb对于每个区域的实际内存量而言,该设置太高了。将此值从 32MiB 降低到 1MiB。

答案2

编辑:这个答案完全没有抓住问题。请参阅 Matthew Ife 关于 vm.min_free_kbytes 的疯狂高设置的帖子

路由器tor已经运行了一段时间了,对吗?

我怀疑您受到了外部(内存)碎片的影响。如果您查看 OOM 日志,您会发现有很多可用内存块,但如果没有一个块的大小达到要求的大小tor,则将malloc()失败。

Normal: 465*4kB (UEM) 285*8kB (UEM) 219*16kB (UEM) 92*32kB (UEM) 51*64kB (UEM) 25*128kB (UEM) 12*256kB (UEM) 2*512kB (U) 0*1024kB 0*2048kB 2*4096kB (R) = 29340kB
HighMem: 5997*4kB (UM) 2743*8kB (UM) 478*16kB (UM) 27*32kB (UM) 2*64kB (M) 2*128kB (U) 1*256kB (U) 2*512kB (U) 0*1024kB 1*2048kB (U) 1*4096kB (R) = 62252kB

编辑:正如 Matthew 指出的那样,有问题的 malloc() 请求顺序 0(单页)和顺序 1(双页)分配。我显然搞错了方向。

对于发生的事情有一个很好的解释elinux网站。遗憾的是,该演示文稿中提到的内存压缩似乎在 DD-WRT 中不可调整,尽管我可以找到可追溯到的变更集2010这表明它在内核中。

至于你应该做什么,tor我不太清楚。也许你可以为它运行一个看门狗,或者适时重新启动该进程(这应该有助于内核找到空间来清理内存)。也有可能设置得swappiness太高会减慢内存压缩过程,但我现在只是猜测。

答案3

你有几个明显的问题。

第一的:

vm.swappiness = 100

您已经告诉内核尽可能积极地进行交换,即使没有严格必要这样做。

在嵌入式设备上(通常根本没有交换),这完全是疯了。应该将其设置为 0(可能是最好的)或 1。

第二:

vm.overcommit_memory = 2

除了完全防止过度使用,并防止系统使用所有可用内存和交换之外,这admin_reserve_kbytes导致为和中指定的管理员和用户进程保留额外的内存user_reserve_kbytes

我找不到我管理的单个系统,无论是嵌入式、桌面还是服务器,其此设置都不为 0。

最后,关于 DD-WRT...我对他们不满意。他们前段时间过度商业化,停止了定期发布和更新。现在我推荐 OpenWrt 作为构建嵌入式系统的平台。

相关内容