Python Multiprocessing - 将类方法应用于对象列表

2022-01-12 00:00:00 python multiprocessing

问题描述

有没有一种简单的方法可以使用多处理来完成此操作?

Is there a simple way to use Multiprocessing to do the equivalent of this?

for sim in sim_list:
  sim.run()

其中 sim_list 的元素是模拟"对象，而 run() 是模拟类的一个方法，它确实修改对象的属性.例如:

where the elements of sim_list are "simulation" objects and run() is a method of the simulation class which does modify the attributes of the objects. E.g.:

class simulation:
    def __init__(self):
        self.state['done']=False
        self.cmd="program"
    def run(self):
        subprocess.call(self.cmd)
        self.state['done']=True

sim_list 中的所有 sim 都是独立的，因此策略不必是线程安全的.

All the sim in sim_list are independent, so the strategy does not have to be thread safe.

我尝试了以下，这显然是有缺陷的，因为参数是通过 deepcopy 传递的，并且没有就地修改.

I tried the following, which is obviously flawed because the argument is passed by deepcopy and is not modified in-place.

from multiprocessing import Process

for sim in sim_list:
  b = Process(target=simulation.run, args=[sim])
  b.start()
  b.join()

解决方案

做你想做的事情的一种方法是让你的计算类(simulation 在你的例子中)成为 的子类处理.正确初始化后，此类的实例将在单独的进程中运行，您可以根据需要从列表中设置一组实例.

One way to do what you want is to have your computing class (simulation in your case) be a subclass of Process. When initialized properly, instances of this class will run in separate processes and you can set off a group of them from a list just like you wanted.

这是一个示例，基于您上面写的内容:

Here's an example, building on what you wrote above:

import multiprocessing
import os
import random

class simulation(multiprocessing.Process):
    def __init__(self, name):
        # must call this before anything else
        multiprocessing.Process.__init__(self)

        # then any other initialization
        self.name = name
        self.number = 0.0
        sys.stdout.write('[%s] created: %f
' % (self.name, self.number))

    def run(self):
        sys.stdout.write('[%s] running ...  process id: %s
' 
                         % (self.name, os.getpid()))

        self.number = random.uniform(0.0, 10.0)
        sys.stdout.write('[%s] completed: %f
' % (self.name, self.number))

然后只需制作一个对象列表并以循环开始每个对象:

Then just make a list of objects and start each one with a loop:

sim_list = []
sim_list.append(simulation('foo'))
sim_list.append(simulation('bar'))

for sim in sim_list:
    sim.start()

当您运行它时，您应该会看到每个对象都在其自己的进程中运行.不要忘记调用 Process.__init__(self) 作为类初始化中的第一件事，然后再进行其他操作.

When you run this you should see each object run in its own process. Don't forget to call Process.__init__(self) as the very first thing in your class initialization, before anything else.

显然我没有在这个例子中包含任何进程间通信；如果您的情况需要，您必须添加它(从您的问题中不清楚您是否需要它).

Obviously I've not included any interprocess communication in this example; you'll have to add that if your situation requires it (it wasn't clear from your question whether you needed it or not).

这种方法对我很有效，我不知道有什么缺点.如果有人知道我忽略的隐患，请告诉我.

This approach works well for me, and I'm not aware of any drawbacks. If anyone knows of hidden dangers which I've overlooked, please let me know.

我希望这会有所帮助.

相关文章