PyInstaller 构建的 Windows EXE 因多处理而失败

2022-01-12 00:00:00 python python-2.7 pyinstaller windows multiprocessing

问题描述

在我的项目中，我使用 Python 的 multiprocessing 库在 __main__ 中创建多个进程.该项目正在使用 PyInstaller 2.1.1 打包到单个 Windows EXE 中.

In my project I'm using Python's multiprocessing library to create multiple processes in __main__. The project is being packaged into a single Windows EXE using PyInstaller 2.1.1.

我像这样创建新流程:

from multiprocessing import Process
from Queue import Empty

def _start():
    while True:
        try:
            command = queue.get_nowait()
        # ... and some more code to actually interpret commands
        except Empty:
            time.sleep(0.015)

def start():
    process = Process(target=_start, args=args)
    process.start()
    return process

在 __main__ 中:

And in __main__:

if __name__ == '__main__':
    freeze_support()

    start()

不幸的是，在将应用程序打包成 EXE 并启动它时，我在这一行得到 WindowsError 5 或 6(似乎是随机的):

Unfortunately, when packaging the application into an EXE and launching it, I get WindowsError 5 or 6 (seems random) at this line:

command = queue.get_nowait()

PyInstaller 主页上的一个配方声称，在将应用程序打包为单个文件时，我必须修改我的代码以在 Windows 中启用多处理.

A recipe at PyInstaller's homepage claims that I have to modify my code to enable multiprocessing in Windows when packaging the application as a single file.

我在这里复制代码:

import multiprocessing.forking
import os
import sys


class _Popen(multiprocessing.forking.Popen):
    def __init__(self, *args, **kw):
        if hasattr(sys, 'frozen'):
            # We have to set original _MEIPASS2 value from sys._MEIPASS
            # to get --onefile mode working.
            # Last character is stripped in C-loader. We have to add
            # '/' or '\' at the end.
            os.putenv('_MEIPASS2', sys._MEIPASS + os.sep)
        try:
            super(_Popen, self).__init__(*args, **kw)
        finally:
            if hasattr(sys, 'frozen'):
                # On some platforms (e.g. AIX) 'os.unsetenv()' is not
                # available. In those cases we cannot delete the variable
                # but only set it to the empty string. The bootloader
                # can handle this case.
                if hasattr(os, 'unsetenv'):
                    os.unsetenv('_MEIPASS2')
                else:
                    os.putenv('_MEIPASS2', '')


class Process(multiprocessing.Process):
    _Popen = _Popen


class SendeventProcess(Process):
    def __init__(self, resultQueue):
        self.resultQueue = resultQueue

        multiprocessing.Process.__init__(self)
        self.start()

    def run(self):
        print 'SendeventProcess'
        self.resultQueue.put((1, 2))
        print 'SendeventProcess'


if __name__ == '__main__':
    # On Windows calling this function is necessary.
    if sys.platform.startswith('win'):
        multiprocessing.freeze_support()
    print 'main'
    resultQueue = multiprocessing.Queue()
    SendeventProcess(resultQueue)
    print 'main'

我对这个解决方案"的失望在于，第一，完全不清楚它到底在修补什么，第二，它的编写方式如此复杂，以至于无法推断出哪些部分是解决方案，哪些部分是解决方案.只是一个插图.

My frustration with this "solution" is that, one, it's absolutely unclear what exactly it is patching, and, two, that it's written in such a convoluted way that it becomes impossible to infer which parts are the solution, and which are just an illustration.

任何人都可以就这个问题分享一些观点，并提供见解在一个项目中究竟需要更改哪些内容，以便在 PyInstaller 构建的单文件 Windows 可执行文件中启用多处理?

Can anyone share some light on this issue, and provide insight what exactly needs to be changed in a project that enables multiprocessing in PyInstaller-built single-file Windows executables?

解决方案

找到这个PyInstaller后回答我自己的问题票:

显然我们所要做的就是提供一个如下所示的Process(和_Popen)类，并使用它来代替multiprocessing.Process.我已更正并简化了该类，使其仅适用于 Windows，*ix 系统可能需要不同的代码.

Apparently all we have to do is provide a Process (and _Popen) class as shown below, and use it instead of multiprocessing.Process. I've corrected and simplified the class to work on Windows only, *ix systems might need different code.

为了完整起见，以下是上述问题的改编样本:

For the sake of completeness, here's the adapted sample from the above question:

import multiprocessing
from Queue import Empty

class _Popen(multiprocessing.forking.Popen):
    def __init__(self, *args, **kw):
        if hasattr(sys, 'frozen'):
            os.putenv('_MEIPASS2', sys._MEIPASS)
        try:
            super(_Popen, self).__init__(*args, **kw)
        finally:
            if hasattr(sys, 'frozen'):
                os.unsetenv('_MEIPASS2')


class Process(multiprocessing.Process):
    _Popen = _Popen


def _start():
    while True:
        try:
            command = queue.get_nowait()
        # ... and some more code to actually interpret commands
        except Empty:
            time.sleep(0.015)

def start():
    process = Process(target=_start, args=args)
    process.start()
    return process

相关文章