使用具有多个线程的rabbitmq消息队列(Python Kombu)
问题描述
我有一个带有单个队列的 RabbitMQ 交换.我希望创建一个运行多个线程并尽快通过该队列工作的守护进程.
I have a single RabbitMQ exchange with a single queue. I wish to create a daemon that runs multiple threads and works through this queue as quickly as possible.
工作"涉及与外部服务的通信，因此每个消费者内部都会发生相当多的阻塞.因此，我希望多个线程都处理来自同一个队列的消息.
The "work" involves communicating with external services, so there will be a fair amount of blocking going on within each consumer. As such, I want to have multiple threads all dealing with messages from the same queue.
我可以通过使用主线程上的队列，然后将传入的工作分配给其他线程池来实现这一点，但是有没有办法在各自的线程上下文中启动多个消费者?
I can achieve this by consuming the queue on my primary thread, and then farming the incoming work off to a pool of other threads, but is there a way to launch multiple consumers, each within their own threaded context?
解决方案
看看 celery - 它是专门设计的排队和处理来自 AMPQ 代理的任务(但也适用于其他排队后端).它处理多进程或多线程并发，并使得创建和使用任务变得非常容易.
Take a look at celery - it is designed to queue and process tasks from an AMPQ broker (but also works with other queuing backends). It handles multiprocess or multithreaded concurrency and makes it very easy to create and consume tasks.
相关文章