新手 python 子进程:“写入错误:管道损坏"

2022-01-18 00:00:00 python subprocess popen

问题描述

感谢以下有用的建议:

所以当我修复它时似乎是固定的

So it seems to be fixed when I

  1. 将命令分离到对 Popen 的单独调用中
  2. stderr=subprocess.PIPE 作为每个 Popen 链的参数.

新代码:

import subprocess
import shlex
import logging

def run_shell_commands(cmds):
    """ Run commands and return output from last call to subprocess.Popen.
        For usage see the test below.
    """
    # split the commands
    cmds = cmds.split("|")
    cmds = list(map(shlex.split,cmds))

    logging.info('%s' % (cmds,))

    # run the commands
    stdout_old = None
    stderr_old = None
    p = []
    for cmd in cmds:
        logging.info('%s' % (cmd,))
        p.append(subprocess.Popen(cmd,stdin=stdout_old,stdout=subprocess.PIPE,stderr=subprocess.PIPE))
        stdout_old = p[-1].stdout
        stderr_old = p[-1].stderr
    return p[-1]


pattern = '"^85567      "'
file = "j"

cmd1 = 'grep %s %s | sort -g -k3 | head -10 | cut -d" " -f2,3' % (pattern, file)
p = run_shell_commands(cmd1)
out = p.communicate()
print(out)

原帖:

我花了太长时间试图解决一个简单的 subprocess.Popen 管道问题.

I've spent too long trying to solve a problem piping a simple subprocess.Popen.

代码:

import subprocess
cmd = 'cat file | sort -g -k3 | head -20 | cut -f2,3' % (pattern,file)
p = subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE)
for line in p.stdout:
    print(line.decode().strip())

文件的输出长度约为 1000 行:

Output for file ~1000 lines in length:

...
sort: write failed: standard output: Broken pipe
sort: write error

长度>241 行的文件的输出:

Output for file >241 lines in length:

...
sort: fflush failed: standard output: Broken pipe
sort: write error

<241 行长度的文件的输出很好.

Output for file <241 lines in length is fine.

我一直在疯狂地阅读文档和谷歌搜索，但是我缺少关于子进程模块的一些基本内容......可能与缓冲区有关.我试过 p.stdout.flush() 并使用缓冲区大小和 p.wait().我试图用像'sleep 20;'这样的命令来重现它.catmoderatefile' 但这似乎运行没有错误.

I have been reading the docs and googling like mad but there is something fundamental about the subprocess module that I'm missing ... maybe to do with buffers. I've tried p.stdout.flush() and playing with the buffer size and p.wait(). I've tried to reproduce this with commands like 'sleep 20; cat moderatefile' but this seems to run without error.

解决方案

来自 子流程 文档:

# To replace shell pipeline like output=`dmesg | grep hda`
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
output = p2.communicate()[0]

相关文章