如何杀死多进程中的所有池工作人员?

2022-01-12 00:00:00 python multithreading multiprocessing kill-process

问题描述

我想停止单个工作人员的所有线程.

I want to stop all threads from a single worker.

我有一个有 10 个工人的线程池:

I have a thread pool with 10 workers:

def myfunction(i):
    print(i) 
    if (i == 20):
        sys.exit()

p = multiprocessing.Pool(10, init_worker) 

for i in range(100):
    p.apply_async(myfunction, (i,))

我的程序不会停止，其他进程会继续工作，直到完成所有 100 次迭代.我想完全从调用 sys.exit() 的线程内部停止池.目前的写法只会停止调用sys.exit()的worker.

My program does not stop and the other processes continue working until all 100 iterations are complete. I want to stop the pool entirely from inside the thread that calls sys.exit(). The way it is currently written will only stop the worker that calls sys.exit().

解决方案

这不是你想要的方式，因为在工作进程中调用 sys.exit() 只会终止工人.它对父进程或其他工作进程没有影响，因为它们是独立的进程，提高 SystemExit 只会影响当前进程.您需要向父进程发送一个信号，告诉它应该关闭.为您的用例执行此操作的一种方法是使用 Event 在 multiprocessing.Manager 服务器:

This isn't working the way you're intending because calling sys.exit() in a worker process will only terminate the worker. It has no effect on the parent process or the other workers, because they're separate processes and raising SystemExit only affects the current process. You need to send a signal back the parent process to tell it that it should shut down. One way to do this for your use-case would be to use an Event created in a multiprocessing.Manager server:

import multiprocessing

def myfunction(i, event):
    if not event.is_set():
        print i 
    if i == 20:
        event.set()

if __name__ == "__main__":
    p= multiprocessing.Pool(10) 
    m = multiprocessing.Manager()
    event = m.Event()
    for i in range(100):
        p.apply_async(myfunction , (i, event))
    p.close()

    event.wait()  # We'll block here until a worker calls `event.set()`
    p.terminate() # Terminate all processes in the Pool

输出:

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

正如 Luke 的回答中所指出的，这里有一场竞赛:不能保证所有工作人员都会按顺序运行，因此 myfunction(20, ..) 可能会在之前运行例如，myfuntion(19, ..).20 之后的其他工作人员也可能在主进程可以对正在设置的事件采取行动之前运行.我通过在打印 i 之前添加 if not event.is_set(): 调用来减小比赛窗口的大小，但它仍然存在.

As pointed out in Luke's answer, there is a race here: There's no guarantee that all the workers will run in order, so it's possible that myfunction(20, ..) will run prior to myfuntion(19, ..), for example. It's also possible that other workers after 20 will run before the main process can act on the event being set. I reduced the size of the race window by adding the if not event.is_set(): call prior to printing i, but it still exists.

相关文章