在 Windows 上导入和使用使用多处理而不会导致无限循环的模块

2022-01-12 00:00:00 python windows multiprocessing

问题描述

我有一个名为 multi.py 的模块.如果我只是想将 multi.py 作为脚本执行，那么避免在 Windows 上崩溃(产生无限数量的进程)的解决方法是将多处理代码放在:

I have a module named multi.py. If I simply wanted to execute multi.py as a script, then the workaround to avoid crashing on Windows (spawning an infinite number of processes) is to put the multiprocessing code under:

if __name__ == '__main__':

但是，我尝试将它作为模块从另一个脚本导入并调用 multi.start().这如何实现?

However, I am trying to import it as a module from another script and call multi.start(). How can this be accomplished?

# multi.py
import multiprocessing

def test(x):
    x**=2

def start():
    pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()-2)
    pool.map(test, (i for i in range(1000*1000)))
    pool.terminate()
    print('done.')

if __name__ == '__main__':
    print('runs as a script,',__name__)
else:
    print('runs as imported module,',__name__)

这是我运行的 test.py:

# test.py
import multi
multi.start()

解决方案

我不太明白你在问什么.你不需要做任何事情来防止它产生无限多的进程.我只是在 Windows XP 上运行它 --- 导入文件并运行 multi.start() --- 它在几秒钟内完成.

I don't quite get what you're asking. You don't need to do anything to prevent this from spawning infinitely many processes. I just ran it on Windows XP --- imported the file and ran multi.start() --- and it completed fine in a couple seconds.

您必须执行 if __name__=="__main__" 保护的原因是，在 Windows 上，多处理必须导入主脚本才能运行目标函数，这意味着顶部-该文件中的级别模块代码将被执行.仅当顶级模块代码本身尝试生成新进程时，才会出现问题.在您的示例中，顶级模块代码不使用多处理，因此没有无限的进程链.

The reason you have to do the if __name__=="__main__" protection is that, on Windows, multiprocessing has to import the main script in order to run the target function, which means top-level module code in that file will be executed. The problem only arises if that top-level module code itself tries to spawn a new process. In your example, the top level module code doesn't use multiprocessing, so there's no infinite process chain.

现在我明白你的要求了.您不需要保护 multi.py.你需要保护你的主脚本，不管它是什么.如果您遇到崩溃，那是因为在您的主脚本中，您在顶级模块代码中执行 multi.start().您的脚本需要如下所示:

Now I get what you're asking. You don't need to protect multi.py. You need to protect your main script, whatever it is. If you're getting a crash, it's because in your main script you are doing multi.start() in the top level module code. Your script needs to look like this:

import multi
if __name__=="__main__":
    multi.start()

main 脚本中始终需要保护".

The "protection" is always needed in the main script.

相关文章