交互式 Python 中的多处理包

2022-01-12 00:00:00 python multiprocessing interactive

问题描述

我有以下代码test.py:

I have the following code test.py:

#multiprocessing in the interactive Python 

import time
from multiprocessing import Process, Pipe

def MyProcess(a):

    while(1):
       time.sleep(1)
       a.send("tic")    

if __name__ == "__main__":

    a, b = Pipe() 

    p = Process(target=MyProcess, args=(a,))
    p.start()

    while(1):
       msg=b.recv()
       print(msg)

如果我在 DOS shell "python test.py" 中执行它就可以正常工作但如果我使用 IEP (Pyzo) 中的执行文件"，它就不起作用.

It works fine if I execute it in the DOS shell "python test.py" But it doesn't work if I use "Execute File" from IEP (Pyzo).

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:pyzo2014a_64blibmultiprocessingspawn.py", line 106, in spawn_main
    exitcode = _main(fd)
  File "C:pyzo2014a_64blibmultiprocessingspawn.py", line 116, in _main
    self = pickle.load(from_parent)
AttributeError: Can't get attribute 'MyProcess' on <module '__main__' (built-in)>

我发现这是一个记录在案的问题".请检查以下链接的答案.

I found that this is a documented 'issue'. Please check the answer of the link below.

交互模式下的多处理中断

这是否意味着我不应该使用交互式 Python 中的多处理包?这是否意味着我无法从 IPython 控制台创建进程?对此的任何澄清将不胜感激

Does it mean that I should not use multiprocessing package from the interactive Python? Does it mean I can not create a process from the IPython console? Any clarification on this will be highly appreciated

解决方案

正确，你不能从解释器中使用 multiprocessing... 主要是因为 pickle 不知道如何序列化交互功能.然而，如果你使用一个名为 pathos.multiprocessing 的 multiprocessing 分支，你可以从解释器中做你想做的事.这是因为 pathos.multiprocessing 使用 dill，而 dill 知道如何序列化解释器中定义的函数(和其他对象).

Correct, you can't use multiprocessing from the interpreter… primarily because pickle doesn't know how to serialize interactive functions. However, if you use a multiprocessing fork, called pathos.multiprocessing, you can do what you want from the interpreter. This works because pathos.multiprocessing uses dill, and dill knows how to serialize functions (and other objects) that are defined in the interpreter.

>>> from pathos.multiprocessing import ProcessingPool as Pool
>>> 
>>> p = Pool(4)
>>> def squared(x):
...   return x**2
... 
>>> def pow(x,y):
...   return x**y
... 
>>> a = range(10)
>>> b = range(10,0,-1)
>>> 
>>> p.map(squared, a)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> res = p.amap(pow, a, b)
>>> print "asynchronous, and with multiple inputs!"
asynchronous, and with multiple inputs!
>>> res.get()
[0, 1, 256, 2187, 4096, 3125, 1296, 343, 64, 9]

在此处获取 pathos:https://github.com/uqfoundation

相关文章