撤销 celery 的任务
问题描述
我想明确撤销 celery 的任务.这就是我目前的做法:-
I want to explicitly revoke a task from celery. This is how I'm currently doing:-
from celery.task.control import revoke
revoke(task_id, terminate=True)
其中task_id是string(也尝试过将其转换为UUID uuid.UUID(task_id).hex).
where task_id is string(have also tried converting it into UUID uuid.UUID(task_id).hex).
经过上述过程，当我再次启动 celery celery worker -A proj 时，它仍然使用相同的消息并开始处理它.为什么?
After the above procedure, when I start celery again celery worker -A proj it still consumes the same message and starts processing it. Why?
当通过 flower 查看时，消息仍然存在于代理部分.如何删除消息，使其无法再次使用?
When viewed via flower, the message is still there in the broker section. how do I delete the message so that it cant be consumed again?
解决方案
revoke是如何工作的?
当调用 revoke 方法时，任务不会立即从队列中删除，它所做的只是告诉 celery(不是你的代理！)保存 task_id在内存 set 中(查看 这里如果你像我一样喜欢阅读源代码).
How does revoke works?
When calling the revoke method the task doesn't get deleted from the queue immediately, all it does is tell celery(not your broker!) to save the task_id in a in-memory set(look here if you like reading source code like me).
当任务到达队列的顶部时，Celery 会检查它是否在撤销集中，如果是，则不会执行.
When the task gets to the top of the queue, Celery will check if is it in the revoked set, if it does, it won't execute it.
它以这种方式工作以防止对每个 revoke 调用进行 O(n) 搜索，其中检查 task_id 是否在内存集中只需 O(1)
It works this way to prevent O(n) search for each revoke call, where checking if the task_id is in the in-memory set is just O(1)
了解事情是如何工作的，你现在知道 set 只是一个普通的 python 集，它被保存在 in-memory - 这意味着当你重新启动时，你会丢失这一套，但任务(当然)是持久化的，当任务轮到时，它会照常执行.
Understanding how things works, you now know that the set is just a normal python set, that being saved in-memory - that means when you restart, you lose this set, but the task is(of course) persistence and when the tasks turn comes, it will be executed as normal.
你需要有一个持久化集，这是通过初始化你的工人来完成的:
You will need to have a persistence set, this is done by initial your worker like this:
celery worker -A proj --statedb=/var/run/celery/worker.state
这会将集合保存在文件系统上.
This will save the set on the filesystem.
参考资料:
- Celery 内存集源代码
- 撤销文档
- 持久撤销文档
相关文章