这些java本机内存是从哪里分配的?
JDK版本是热点8u_45
我研究了我的 java 进程的本机内存.本机内存甚至比堆消耗更多的空间.然而，有许多本机内存块让我感到困惑.pmap -x 的结果例如:
00007f8128000000 65508 25204 25204 rw--- [匿名]00007f812bff9000 28 0 0 ----- [匿名]00007f812c000000 65508 24768 24768 rw--- [匿名]00007f812fff9000 28 0 0 ----- [匿名]00007f8130000000 65508 25532 25532 rw--- [匿名]00007f8133ff9000 28 0 0 ----- [匿名]00007f8134000000 65524 22764 22764 rw--- [匿名]00007f8137ffd000 12 0 0 ----- [匿名]00007f8138000000 65508 26456 26456 rw--- [匿名]00007f813bff9000 28 0 0 ----- [匿名]00007f813c000000 65508 23572 23572 rw--- [匿名]00007f813fff9000 28 0 0 ----- [匿名]00007f8140000000 65520 23208 23208 rw--- [匿名]00007f8143ffc000 16 0 0 ----- [匿名]00007f8144000000 65512 23164 23164 rw--- [匿名]00007f8147ffa000 24 0 0 ----- [匿名]00007f8148000000 65516 23416 23416 rw--- [匿名]00007f814bffb000 20 0 0 ----- [匿名]00007f814c000000 65508 23404 23404 rw--- [匿名]00007f814fff9000 28 0 0 ----- [匿名]00007f8150000000 65512 24620 24620 rw--- [匿名]00007f8153ffa000 24 0 0 ----- [匿名]00007f8154000000 65536 23976 23976 rw--- [匿名]00007f8158000000 65508 23652 23652 rw--- [匿名]00007f815bff9000 28 0 0 ----- [匿名]00007f815c000000 65508 23164 23164 rw--- [匿名]00007f815fff9000 28 0 0 ----- [匿名]00007f8160000000 65508 23344 23344 rw--- [匿名]00007f8163ff9000 28 0 0 ----- [匿名]00007f8164000000 65508 24052 24052 rw--- [匿名]00007f8167ff9000 28 0 0 ----- [匿名]00007f8168000000 131052 48608 48608 rw--- [匿名]00007f816fffb000 20 0 0 ----- [匿名]00007f8170000000 65516 23056 23056 rw--- [匿名]00007f8173ffb000 20 0 0 ----- [匿名]00007f8174000000 65516 26860 26860 rw--- [匿名]00007f8177ffb000 20 0 0 ----- [匿名]00007f8178000000 65508 23360 23360 rw--- [匿名]00007f817bff9000 28 0 0 ----- [匿名]00007f817c000000 65536 24856 24856 rw--- [匿名]00007f8180000000 65512 23272 23272 rw--- [匿名]00007f8183ffa000 24 0 0 ----- [匿名]00007f8184000000 65508 23688 23688 rw--- [匿名]00007f8187ff9000 28 0 0 ----- [匿名]00007f8188000000 65512 24024 24024 rw--- [匿名]00007f818bffa000 24 0 0 ----- [匿名]00007f818c000000 65508 25020 25020 rw--- [匿名]00007f818fff9000 28 0 0 ----- [匿名]00007f8190000000 65512 22868 22868 rw--- [匿名]00007f8193ffa000 24 0 0 ----- [匿名]00007f8194000000 65508 24156 24156 rw--- [匿名]00007f8197ff9000 28 0 0 ----- [匿名]00007f8198000000 65508 23684 23684 rw--- [匿名]有很多块，大约占64M.
我使用 jcmd pid VM.native_memory detail 来跟踪这些内存块.但是，我找不到这些具有 jcmd 结果中列出的任何内存范围的块.
此外，我注意到一篇文章提到了 glic 的 malloc 中的竞技场效果
mem.bin.2
mem.bin.3
mem.bin.4
如图所示，大约有 30 个方块.
几天后，我使用 Google perf 工具来跟踪堆分配.并发现了这个:
说明:zip inflates 消耗近2G内存.我猜它可能与一些编译问题有关.
我看过这个问题:https://bugs.openjdk.java.net/browse/JDK-8164293.这与我的担忧有关吗?
那么如何追踪这些内存块的来源呢?
解决方案使用jemalloc或 tcmalloc - 它们都有内置分配有助于识别分配来源的分析器.
Java 进程可能出于多种原因使用过多的本机内存.受欢迎的原因是
- 直接字节缓冲区
Unsafe.allocateMemory分配的内存- 未关闭的资源(例如
ZipInputStream) - 其他原生库
请注意，NativeMemoryTracking 不会显示本机库消耗的内存.
JDK version is hotspot 8u_45
I researched native memory of my java process. The native memory even consumes more space than heap. However there are many native memory blocks which confuses me. The result of pmap -x for example:
00007f8128000000 65508 25204 25204 rw--- [ anon ]
00007f812bff9000 28 0 0 ----- [ anon ]
00007f812c000000 65508 24768 24768 rw--- [ anon ]
00007f812fff9000 28 0 0 ----- [ anon ]
00007f8130000000 65508 25532 25532 rw--- [ anon ]
00007f8133ff9000 28 0 0 ----- [ anon ]
00007f8134000000 65524 22764 22764 rw--- [ anon ]
00007f8137ffd000 12 0 0 ----- [ anon ]
00007f8138000000 65508 26456 26456 rw--- [ anon ]
00007f813bff9000 28 0 0 ----- [ anon ]
00007f813c000000 65508 23572 23572 rw--- [ anon ]
00007f813fff9000 28 0 0 ----- [ anon ]
00007f8140000000 65520 23208 23208 rw--- [ anon ]
00007f8143ffc000 16 0 0 ----- [ anon ]
00007f8144000000 65512 23164 23164 rw--- [ anon ]
00007f8147ffa000 24 0 0 ----- [ anon ]
00007f8148000000 65516 23416 23416 rw--- [ anon ]
00007f814bffb000 20 0 0 ----- [ anon ]
00007f814c000000 65508 23404 23404 rw--- [ anon ]
00007f814fff9000 28 0 0 ----- [ anon ]
00007f8150000000 65512 24620 24620 rw--- [ anon ]
00007f8153ffa000 24 0 0 ----- [ anon ]
00007f8154000000 65536 23976 23976 rw--- [ anon ]
00007f8158000000 65508 23652 23652 rw--- [ anon ]
00007f815bff9000 28 0 0 ----- [ anon ]
00007f815c000000 65508 23164 23164 rw--- [ anon ]
00007f815fff9000 28 0 0 ----- [ anon ]
00007f8160000000 65508 23344 23344 rw--- [ anon ]
00007f8163ff9000 28 0 0 ----- [ anon ]
00007f8164000000 65508 24052 24052 rw--- [ anon ]
00007f8167ff9000 28 0 0 ----- [ anon ]
00007f8168000000 131052 48608 48608 rw--- [ anon ]
00007f816fffb000 20 0 0 ----- [ anon ]
00007f8170000000 65516 23056 23056 rw--- [ anon ]
00007f8173ffb000 20 0 0 ----- [ anon ]
00007f8174000000 65516 26860 26860 rw--- [ anon ]
00007f8177ffb000 20 0 0 ----- [ anon ]
00007f8178000000 65508 23360 23360 rw--- [ anon ]
00007f817bff9000 28 0 0 ----- [ anon ]
00007f817c000000 65536 24856 24856 rw--- [ anon ]
00007f8180000000 65512 23272 23272 rw--- [ anon ]
00007f8183ffa000 24 0 0 ----- [ anon ]
00007f8184000000 65508 23688 23688 rw--- [ anon ]
00007f8187ff9000 28 0 0 ----- [ anon ]
00007f8188000000 65512 24024 24024 rw--- [ anon ]
00007f818bffa000 24 0 0 ----- [ anon ]
00007f818c000000 65508 25020 25020 rw--- [ anon ]
00007f818fff9000 28 0 0 ----- [ anon ]
00007f8190000000 65512 22868 22868 rw--- [ anon ]
00007f8193ffa000 24 0 0 ----- [ anon ]
00007f8194000000 65508 24156 24156 rw--- [ anon ]
00007f8197ff9000 28 0 0 ----- [ anon ]
00007f8198000000 65508 23684 23684 rw--- [ anon ]
There are many blocks which occupy about 64M.
I use jcmd pid VM.native_memory detail to track these memory blocks. However, I cannot found these blocks with any of the memory ranges listed in the result of jcmd.
Furthermore, I noticed an article which mentions arena effect in malloc of glic Java 8 and Virtual Memory on Linux. However These blocks seem different from thread pool because 1. The mode is rw--- not ----- 2. The arena thread pool only affects virtual memory. It cannot explain these too much RSS.
I use gdb to track the allocated memory
dump binary memory mem.bin from to
mem.bin.1
mem.bin.2
mem.bin.3
mem.bin.4
There are about 30 blocks like those shown in the picture.
After some days, I use Google perf tools to track heap allocations. And found this:
It shows that: zip inflates consume nearly 2G memory. I guess it may concern with some compilation issue.
I have read this issue：https://bugs.openjdk.java.net/browse/JDK-8164293. Is this related to my concern?
So how can I track the source of these memory block?
解决方案Use jemalloc or tcmalloc - they both have built-in allocation profiler that will help to identify the source of allocations.
Java process may use too much native memory for many reasons. Popular reasons are
- Direct ByteBuffers
- Memory allocated by
Unsafe.allocateMemory - Unclosed resources (e.g.
ZipInputStream) - other native libraries
Note that NativeMemoryTracking will not show memory consumed by native libraries.
相关文章