Windows上的Python子进程输出?

2022-01-18 00:00:00 python windows subprocess

问题描述

我在从子进程标准输出管道获取输出时遇到了一些困难.我正在通过它启动一些第三方代码，以提取日志输出.直到最近更新第三方代码，一切正常.更新后，python 开始无限期阻塞，实际上并没有显示任何输出.我可以手动启动第三方应用程序并查看输出.

I'm running into some difficulties getting output from a subprocess stdout pipe. I'm launching some third party code via it, in order to extract log output. Up until a recent update of the third party code, everything worked fine. After the update, python has started blocking indefinitely, and not actually showing any output. I can manually launch the third party app fine and see output.

我正在使用的代码的基本版本:

A basic version of the code I'm using:

import subprocess, time
from threading import Thread

def enqueue_output(out):
    print "Hello from enqueue_output"
    for line in iter(out.readline,''):
        line = line.rstrip("
")
        print "Got %s" % line
    out.close()

proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, bufsize=1)
thread = Thread(target=enqueue_output, args=(proc.stdout,))
thread.daemon = True
thread.start()

time.sleep(30)

如果我用 third_party.exe 代替这个脚本，这将完美运行:

This works perfectly if I substitute third_party.exe for this script: 

import time, sys

while True:
    print "Test"
    sys.stdout.flush()
    time.sleep(1)

所以我不清楚需要做些什么魔法才能让这个与原始命令一起工作.

So I'm unclear as to magic needs to be done to get this working with the original command.

这些都是 subprocess.Popen 行的变体，我尝试过但没有成功:

These are all variants of the subprocess.Popen line I've tried with no success:

proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, bufsize=0)
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, shell=True)
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, creationflags=subprocess.CREATE_NEW_CONSOLE)
si = subprocess.STARTUPINFO()
si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, startupinfo=si)

编辑 1:在这种情况下，我实际上不能使用 .communicate() .我正在启动的应用程序会长时间运行(几天到几周).我可以实际测试 .communicate() 的唯一方法是在应用程序启动后不久将其终止，我认为这不会给我有效的结果.

Edit 1: I can't actually use .communicate() in this case. The app I'm launching remains running for long periods of time (days to weeks). The only way I could actually test .communicate() would be to kill the app shortly after it launches, which I don't feel would give me valid results.

即使是非线程版本也失败了:

Even the non-threaded version of this fails:

import subprocess, time
from threading import Thread

proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, stderr=subprocess.PIPE)

print "App started, reading output..."
for line in iter(proc.stdout.readline,''):
    line = line.rstrip("
")
    print "Got: %s" % line

编辑 2:感谢 jdi，以下工作正常:

Edit 2: Thanks to jdi, the following works okay:

import tempfile, time, subprocess

w = "test.txt"
f = open("test.txt","a")
p = subprocess.Popen("third_party.exe", shell=True, stdout=f,
                        stderr=subprocess.STDOUT, bufsize=0)

time.sleep(30)

with open("test.txt", 'r') as r:
    for line in r:
        print line
f.close()

解决方案

首先我建议你简化这个例子，以确保你可以真正阅读任何内容.从混合中删除线程的复杂性:

First I would recommend that you simplify this example to make sure you can actually read anything. Remove the complication of the thread from the mix:

proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, bufsize=1)
print proc.communicate()

如果可行，那就太好了.那么您可能在如何直接或可能在您的线程中读取标准输出时遇到问题.

If that works, great. Then you are having problems possibly with how you are reading the stdout directly or possibly in your thread.

如果这不起作用，您是否尝试过将 stderr 也通过管道传输到 stdout?

If this does not work, have you tried piping stderr to stdout as well?

proc = subprocess.Popen("third_party.exe", 
                        stdout=subprocess.PIPE, 
                        stderr=subprocess.STDOUT, bufsize=1)

更新

既然你说 communicate() 是死锁的，这里有另一种方法你可以试试看是不是子进程内部缓冲区的问题...

Since you say communicate() is deadlocking, here is another approach you can try to see if its a problem with the internal buffer of subprocess...

import tempfile
import subprocess

w = tempfile.NamedTemporaryFile()
p = subprocess.Popen('third_party.exe', shell=True, stdout=w, 
                        stderr=subprocess.STDOUT, bufsize=0)

with open(w.name, 'r') as r:
    for line in r:
        print line
w.close()

相关文章