Python 多进程池.当其中一个工作进程确定不再需要完成工作时，如何退出脚本?
问题描述
mp.set_start_method('spawn')
total_count = Counter(0)
pool = mp.Pool(initializer=init, initargs=(total_count,), processes=num_proc)
pool.map(part_crack_helper, product(seed_str, repeat=4))
pool.close()
pool.join()
所以我有一个工作进程池来做一些工作.它只需要找到一种解决方案.因此，当其中一个工作进程找到解决方案时，我想停止一切.
So I have a pool of worker process that does some work. It just needs to find one solution. Therefore, when one of the worker processes finds the solution, I want to stop everything.
我想到的一种方法就是调用 sys.exit().但是，这似乎无法正常工作，因为其他进程正在运行.
One way I thought of was just calling sys.exit(). However, that doesn't seem like it's working properly since other processes are running.
另一种方法是检查每个进程调用的返回值(part_crack_helper 函数的返回值)并在该进程上调用终止.但是，我不知道在使用该地图功能时该怎么做.
One other way was to check for the return value of each process calls (the return value of part_crack_helper function) and call terminate on that process. However, I don't know how to do that when using that map function.
我应该如何做到这一点?
How should I achieve this?
解决方案
您可以使用来自 Pool.apply_async 的回调.
You can use callbacks from Pool.apply_async.
这样的事情应该可以为您完成这项工作.
Something like this should do the job for you.
from multiprocessing import Pool
def part_crack_helper(args):
solution = do_job(args)
if solution:
return True
else:
return False
class Worker():
def __init__(self, workers, initializer, initargs):
self.pool = Pool(processes=workers,
initializer=initializer,
initargs=initargs)
def callback(self, result):
if result:
print("Solution found! Yay!")
self.pool.terminate()
def do_job(self):
for args in product(seed_str, repeat=4):
self.pool.apply_async(part_crack_helper,
args=args,
callback=self.callback)
self.pool.close()
self.pool.join()
print("good bye")
w = Worker(num_proc, init, [total_count])
w.do_job()
相关文章