多处理启动太多 Python VM 实例

2022-01-12 00:00:00 python windows multiprocessing

问题描述

我正在编写一些多处理代码(Python 2.6.4、WinXP)来生成进程以运行后台任务.在玩一些琐碎的例子时，我遇到了一个问题，即我的代码只是不断产生新的进程，即使我只告诉它产生一个固定的数字.

I am writing some multiprocessing code (Python 2.6.4, WinXP) that spawns processes to run background tasks. In playing around with some trivial examples, I am running into an issue where my code just continuously spawns new processes, even though I only tell it to spawn a fixed number.

程序本身运行良好，但如果我查看 Windows 任务管理器，我不断看到新的python.exe"进程出现.随着程序的运行(最终使我的机器挨饿)，它们只会越来越多地产生.

The program itself runs fine, but if I look in Windows TaskManager, I keep seeing new 'python.exe' processes appear. They just keep spawning more and more as the program runs (eventually starving my machine).


例如，
我希望下面的代码能够启动 2 个 python.exe 进程.第一个是程序本身，第二个是它产生的子进程.知道我做错了什么吗?


For example,
I would expect the code below to launch 2 python.exe processes. The first being the program itself, and the second being the child process it spawns. Any idea what I am doing wrong?

import time
import multiprocessing


class Agent(multiprocessing.Process):
    def __init__(self, i):
        multiprocessing.Process.__init__(self)
        self.i = i

    def run(self):
        while True:
            print 'hello from %i' % self.i
            time.sleep(1)


agent = Agent(1)
agent.start()

解决方案

您似乎没有仔细遵循文档中的指南，特别是 本节 讨论安全导入主模块".

It looks like you didn't carefully follow the guidelines in the documentation, specifically this section where it talks about "Safe importing of main module".

您需要使用 if __name__ == '__main__': 块来保护您的启动代码，否则我相信您会得到您所得到的.

You need to protect your launch code with an if __name__ == '__main__': block or you'll get what you're getting, I believe.

我认为这归结为多处理模块无法像在 Linux 上那样使用 os.fork()，在 Linux 中，已经运行的进程基本上克隆在内存中.在 Windows(没有这样的 fork())上，它必须运行一个新的 Python 解释器并告诉它导入你的主模块，然后在完成后执行 start/run 方法.如果您有模块级别"的代码，不受名称检查的保护，那么在导入过程中它会重新开始整个序列，无穷无尽

I believe it comes down to the multiprocessing module not being able to use os.fork() as it does on Linux, where an already-running process is basically cloned in memory. On Windows (which has no such fork()) it must run a new Python interpreter and tell it to import your main module and then execute the start/run method once that's done. If you have code at "module level", unprotected by the name check, then during the import it starts the whole sequence over again, ad infinitum

相关文章