LinkedBlockingQueue 的 Java 性能问题

2022-01-21 00:00:00 multithreading queue performance java

这是我在stackoverflow上的第一篇文章...希望有人能帮助我

this is my first post on stackoverflow... I hope someone can help me

我对 Java 6 LinkedBlockingQueue 有很大的性能下降.在第一个线程中，我生成了一些我推入队列的对象在第二个线程中，我将这些对象拉出.当 LinkedBlockingQueue 的 take() 方法被频繁调用时，就会出现性能回归.我监控了整个程序，take() 方法占用的时间最多.吞吐量从 ~58Mb/s 到 0.9Mb/s...

I have a big performance regression with Java 6 LinkedBlockingQueue. In the first thread i generate some objects which i push in to the queue In the second thread i pull these objects out. The performance regression occurs when the take() method of the LinkedBlockingQueue is called frequently. I monitored the whole program and the take() method claimed the most time overall. And the throughput goes from ~58Mb/s to 0.9Mb/s...

队列 pop 和 take 方法是使用此类中的静态方法调用的

the queue pop and take methods ar called with a static method from this class 

public class C_myMessageQueue {

    private static final LinkedBlockingQueue<C_myMessageObject> x_queue = new LinkedBlockingQueue<C_myMessageObject>( 50000 );

    /**
     * @param message
     * @throws InterruptedException
     * @throws NullPointerException
     */
    public static void addMyMessage( C_myMessageObject message )
            throws InterruptedException, NullPointerException {
        x_queue.put( message );
    }

    /**
     * @return Die erste message der MesseageQueue
     * @throws InterruptedException
     */
    public static C_myMessageObject getMyMessage() throws InterruptedException {
        return x_queue.take();
    }
}

如何调整 take() 方法以达到至少 25Mb/s，或者我可以使用其他类在队列"满或空时阻塞.

how can I tune the take() method to accomplish at least 25Mb/s, or is there a other class I can use which will block when the "queue" is full or empty.

亲切的问候

巴特

P.S.:抱歉我的英语不好，我来自德国；)

P.S.: sorry for my bad english, I'm from Germany ;)

推荐答案

您的生产者线程只是放入的元素比消费者消耗的多，因此队列最终达到其容量限制，因此生产者等待.

Your producer thread simply puts more elements than the consumer consumes, so the queue eventually hits its capacity limit, thus the producer waits.

巩固我原来的答案，因为现在我们基本上有了全貌:

Consolidating my original answer since now we have basically the full picture:

  • 您通过执行极快的 put() 达到了 LinkedBlockingQueue 的固有吞吐量限制(每个队列都有一个)，其中甚至是连续的 take()s，进一步处理为零，无法跟上.(顺便说一句，这表明在这种结构中，无论如何，在你的 JVM 和机器上，put()s 至少比读取成本略高).
  • 由于消费者锁定了一个特定的锁，因此放置更多的消费者线程可能无济于事(如果您的消费者实际上正在执行一些处理并且限制了吞吐量，那么添加更多消费者会有所帮助.有更好的队列实现在有多个消费者(或生产者)的场景中，您可以尝试 SynchronousQueue、ConcurrentLinkedQueue 以及即将推出的 jsr166y 的 TransferQueue.李>
  • You hit the inherent throughput limit of the LinkedBlockingQueue (every queue has one) by doing extremely fast put()s, where even continual take()s, with zero further processing, cannot keep up. (By the way this shows that in this structure, on your JVM and machine anyway, put()s are at least slightly more costly than the reads).
  • Since there is a particular lock that consumers lock, putting more consumer threads could not possibly help (if your consumer was actually doing some processing and that was bounding the throughput, then adding more consumers would help. There are better queue implementations for a scenario with more than one consumers (or producers), you could try SynchronousQueue, ConcurrentLinkedQueue, and the upcoming TransferQueue of jsr166y).

一些建议:

  • 尝试制作更粗粒度的对象，以便将每个队列的开销与从生产线程卸载的实际工作相平衡(在您的情况下，您似乎为表示可忽略不计的对象创建了很多通信开销工作)
  • 您还可以让生产者通过卸载一些消耗性工作来帮助消费者(当有工作要做时，无所事事地等待没有多大意义).

/在 John W. 正确指出我的原始答案具有误导性之后更新

/updated after John W. rightly pointed out my original answer was misleading

相关文章