问题描述

我决定尝试进入多处理器模块，以帮助加速我的程序。为了解决这个问题，我尝试使用有关多处理的官方python文档中的一些代码示例。

第一次尝试：Introduction

>>> from multiprocessing import Pool
>>>
>>> def f(x):
...     return x*x
...
>>> if __name__ == '__main__':
...     with Pool(5) as p:
...         print(p.map(f, [1, 2, 3]))
...
Process SpawnPoolWorker-3:
Process SpawnPoolWorker-2:
Traceback (most recent call last):
  File "C:Program FilesPython36libmultiprocessingprocess.py", line 258, in _bootstrap
    self.run()
  File "C:Program FilesPython36libmultiprocessingprocess.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "C:Program FilesPython36libmultiprocessingpool.py", line 108, in worker
    task = get()
  File "C:Program FilesPython36libmultiprocessingqueues.py", line 337, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
Traceback (most recent call last):
  File "C:Program FilesPython36libmultiprocessingprocess.py", line 258, in _bootstrap
    self.run()
  File "C:Program FilesPython36libmultiprocessingprocess.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "C:Program FilesPython36libmultiprocessingpool.py", line 108, in worker
    task = get()
  File "C:Program FilesPython36libmultiprocessingqueues.py", line 337, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
Process SpawnPoolWorker-4:
Traceback (most recent call last):
  File "C:Program FilesPython36libmultiprocessingprocess.py", line 258, in _bootstrap
    self.run()
  File "C:Program FilesPython36libmultiprocessingprocess.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "C:Program FilesPython36libmultiprocessingpool.py", line 108, in worker
    task = get()
  File "C:Program FilesPython36libmultiprocessingqueues.py", line 337, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>

在这里，我假设池函数已损坏；可能在最新版本中的某个地方有拼写错误。所以我尝试了一些更具体的代码。

第二次尝试：Process class code block 2

>>> from multiprocessing import Process
>>> import os
>>>
>>> def info(title):
...     print(title)
...     print('module name:', __name__)
...     print('parent process:', os.getppid())
...     print('process id:', os.getpid())
...
>>> def f(name):
...     info('function f')
...     print('hello', name)
...
>>> if __name__ == '__main__':
...     info('main line')
...     p = Process(target=f, args=('bob',))
...     p.start()
...     p.join()
...
main line
module name: __main__
parent process: 43824
process id: 54888
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:Program FilesPython36libmultiprocessingspawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:Program FilesPython36libmultiprocessingspawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>

在这一点上，我知道潜在的错误与多处理的进程函数有关。但是，我认为扩展代码可能破坏了某些东西，所以我尝试使用简单代码。

第三次尝试Process class code block 1

>>> from multiprocessing import Process
>>>
>>> def f(name):
...     print('hello', name)
...
>>> if __name__ == '__main__':
...     p = Process(target=f, args=('bob',))
...     p.start()
...     p.join()
...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:Program FilesPython36libmultiprocessingspawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:Program FilesPython36libmultiprocessingspawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>

在这一点上，我绝望了。我认为参数可能干扰了Process类。

最后一次尝试：自生成代码

>>> from multiprocessing import Process
>>>
>>> def f():
...     print('hello')
...
>>> if __name__ == '__main__':
...     p = Process(target=f)
...     p.start()
...     p.join()
...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:Program FilesPython36libmultiprocessingspawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:Program FilesPython36libmultiprocessingspawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>

现在我完全糊涂了，因为我不知道为什么会发生错误。有人能帮我弄清楚为什么每次都会出现此错误吗？

---

解决方案

您处于交互模式。这基本上不适用于multiprocessing，因为工作人员必须导入__main__并获得与主进程的__main__最相似的内容。这是multiprocessingAPI造成严重混乱的众多方式之一。

将代码放入脚本中，然后运行该脚本。