使用 NIO 直接缓冲区时设置 -XX:+DisableExplicitGC 的影响

2022-01-16 00:00:00 garbage-collection java nio

我们正在构建一个具有激进性能 SLA 的 Web 应用程序，由于 System.gc() 调用导致 JVM 出去吃午饭，这些 SLA 会定期被违反.我们进行了一些调试，并确定在所有情况下，调用 System.gc() 的都是内部应用服务器代码.这在应用服务器启动或应用程序部署时会发生几次，我们并不关心.但是，System.gc() 也会在应用启动并通过内部应用服务器调用 NIO 类运行时定期触发.以下是我们能够捕获此事件的堆栈跟踪:

We are building a web application with aggressive performance SLAs which are periodically being violated due to the JVM going out to lunch due to System.gc() calls. We've done some debugging, and determined that in all cases it's the internal app server code which is invoking System.gc(). This occurs a few times while the app server is booting or the application is deployed, which we aren't concerned about. However, System.gc() is also periodically be triggered when the app is up and running via internal app server calls the NIO classes. Here's a stack trace we were able to capture of this event:

3XMTHREADINFO "WebContainer : 25" J9VMThread:0x0000000006FC5D00, j9thread_t:0x00007F60E41753E0, java/lang/Thread:0x000000060B735590, state:R, prio=5 3XMJAVALTHREAD (java/lang/Thread getId:0xFE, isDaemon:true) 3XMTHREADINFO1 (native thread ID:0x1039, native priority:0x5, native policy:UNKNOWN) 3XMTHREADINFO2 (native stack address range from:0x00007F6067621000, to:0x00007F6067662000, size:0x41000) 3XMCPUTIME CPU usage total: 80.222215853 secs 3XMHEAPALLOC Heap bytes allocated since last GC cycle=1594568 (0x1854C8) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at java/lang/System.gc(System.java:329) 4XESTACKTRACE at java/nio/Bits.syncReserveMemory(Bits.java:721) 5XESTACKTRACE (entered lock: java/nio/Bits@0x000000060000B690, entry count: 1) 4XESTACKTRACE at java/nio/Bits.reserveMemory(Bits.java:766(Compiled Code)) 4XESTACKTRACE at java/nio/DirectByteBuffer.<init>(DirectByteBuffer.java:123(Compiled Code)) 4XESTACKTRACE at java/nio/ByteBuffer.allocateDirect(ByteBuffer.java:306(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/buffermgmt/impl/WsByteBufferPoolManagerImpl.allocateBufferDirect(WsByteBufferPoolManagerImpl.java:706(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/buffermgmt/impl/WsByteBufferPoolManagerImpl.allocateCommon(WsByteBufferPoolManagerImpl.java:612(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/buffermgmt/impl/WsByteBufferPoolManagerImpl.allocateDirect(WsByteBufferPoolManagerImpl.java:527(Compiled Code)) 4XESTACKTRACE at com/ibm/io/async/ResultHandler.runEventProcessingLoop(ResultHandler.java:507(Compiled Code)) 4XESTACKTRACE at com/ibm/io/async/ResultHandler$2.run(ResultHandler.java:905(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1864(Compiled Code)) 3XMTHREADINFO3 Native callstack: 4XENATIVESTACK (0x00007F61083DD122 [libj9prt26.so+0x13122]) 4XENATIVESTACK (0x00007F61083EA79F [libj9prt26.so+0x2079f]) ....

是否有人知道如果我们通过启用 -XX:+DisableExplicitGC 关闭对 System.gc() 的调用会产生什么影响(或者实际上在我们的例子中通过设置 -Xdisableexplicitgc 因为我们在 IBM 上运行 WebsphereJRE，它做同样的事情)?我们当然不想造成内存泄漏.我无法找到直接参考为什么 NIO 中的 System.gc() 调用实际上是必要的，也没有代码注释专门解决它在 JDK 代码中出现的位置:http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/4a1e42601d61/src/share/classes/java/nio/Bits.java

Is anyone aware of what the impact would be if we shut down the calls to System.gc() by enabling -XX:+DisableExplicitGC (or actually in our case by setting -Xdisableexplicitgc since we're running Websphere on the IBM JRE, which does the same thing)? We certainly don't want to create a memory leak. I haven't been able to find a direct reference as to why the System.gc() calls in NIO are actually necessary, and there isn't a code comment specifically addressing it where it occurs in JDK code, either: http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/4a1e42601d61/src/share/classes/java/nio/Bits.java

如果由于使用 NIO 而完全禁用 System.gc() 是个坏主意，那么至少我们可以做些什么来降低调用它的频率吗?看来我们可以设置 -XX:MaxDirectMemorySize，但这似乎只会设置分配内存量的上限，并且可能会产生不利影响.

If it's a bad idea to completely disable System.gc() due to the use of NIO, is there at least something we can do to reduce the frequency at which it's called? It appears that we can set -XX:MaxDirectMemorySize, but this appears as if it would only set up upper bound on the amount of allocated memory, and would just as likely have an adverse affect.

推荐答案

禁用显式 GC 不会阻止缓冲区以及它们持有的本机内存被收集.但它可能会延迟很长一段时间的收集.

Disabling explicit GCs does not prevent buffers and thus the native memory they hold onto from being collected. But it may delay collections for a long time.

这意味着直接缓冲区分配的内存在被收集之前可能会累积很长时间.从长远来看，这并不是真正的泄漏，但会增加峰值内存使用量.

Which means memory allocated by direct buffers may accumulate for a long time before it is collected. In the long run that's not really a leak, but it will increase peak memory usage.

http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/4a1e42601d61/src/share/classes/java/nio/Bits.java

据我了解，当 reserveMemory 限制达到时，System.gc() 调用是为了释放缓冲区.保留请求的数量后，ByteBuffer.allocateDirect 将调用 Unsafe.allocateMemory，它可能会进行自己的 GC 调用，这不应受 DisableExplicitGC 的影响，如果它尝试映射失败.

As I understand it that System.gc() call is there to free buffers when the reserveMemory limit is hit. After reserving the requested amount ByteBuffer.allocateDirect will call to Unsafe.allocateMemory which might do its own GC call, which should not be affected by DisableExplicitGC, if its attempt to mmap fails.

我们至少可以做些什么来减少调用它的频率吗?

is there at least something we can do to reduce the frequency at which it's called?

只有在达到 MaxDirectMemorySize 限制时才会调用它.如果您可以调整您的 GC 或应用程序代码，使其满足以下选项之一:

It is only called when the MaxDirectMemorySize limitation is hit. If you can tune your GC or application code so that it meets one of the following options:

它使用一组固定的缓冲区(-> 永远不会超过限制)
缓冲区被提早收集(短暂的缓冲区 -> 在年轻的 GC 中死亡)
在直接缓冲区空间用完之前定期收集老年代
使用堆缓冲区而不是直接缓冲区

那么 System.gc() 调用就没有必要了.

then System.gc() calls won't be necessary.

在热点上还存在一个 ExplicitGCInvokesConcurrent 选项.也许 IBM 的 VM 也有类似的东西.

On hotspot there also exists a ExplicitGCInvokesConcurrent option. Maybe IBM's VM has something similar.

相关文章