将子进程输出显示到标准输出并重定向它
问题描述
我正在通过 Python 的子进程模块运行脚本.目前我使用:
I'm running a script via Python's subprocess module. Currently I use:
p = subprocess.Popen('/path/to/script', stdout=subprocess.PIPE, stderr=subprocess.PIPE)
result = p.communicate()
然后我将结果打印到标准输出.这一切都很好,但是由于脚本需要很长时间才能完成,所以我也希望从脚本实时输出到标准输出.我管道输出的原因是因为我想解析它.
I then print the result to the stdout. This is all fine but as the script takes a long time to complete, I wanted real time output from the script to stdout as well. The reason I pipe the output is because I want to parse it.
解决方案
将子进程的标准输出保存到变量中以供进一步处理和 在子进程到达时运行时显示它:
To save subprocess' stdout to a variable for further processing and to display it while the child process is running as it arrives:
#!/usr/bin/env python3
from io import StringIO
from subprocess import Popen, PIPE
with Popen('/path/to/script', stdout=PIPE, bufsize=1,
universal_newlines=True) as p, StringIO() as buf:
for line in p.stdout:
print(line, end='')
buf.write(line)
output = buf.getvalue()
rc = p.returncode
保存子进程的 stdout 和 stderr 更复杂,因为您应该同时使用两个流以避免死锁:
To save both subprocess's stdout and stderr is more complex because you should consume both streams concurrently to avoid a deadlock:
stdout_buf, stderr_buf = StringIO(), StringIO()
rc = teed_call('/path/to/script', stdout=stdout_buf, stderr=stderr_buf,
universal_newlines=True)
output = stdout_buf.getvalue()
...
teed_call()
在这里定义.
where teed_call()
is define here.
更新:这里是一个更简单的asyncio
版本.
旧版本:
这是一个基于 child_process 的单线程解决方案.py
示例来自 tulip
:
Here's a single-threaded solution based on child_process.py
example from tulip
:
import asyncio
import sys
from asyncio.subprocess import PIPE
@asyncio.coroutine
def read_and_display(*cmd):
"""Read cmd's stdout, stderr while displaying them as they arrive."""
# start process
process = yield from asyncio.create_subprocess_exec(*cmd,
stdout=PIPE, stderr=PIPE)
# read child's stdout/stderr concurrently
stdout, stderr = [], [] # stderr, stdout buffers
tasks = {
asyncio.Task(process.stdout.readline()): (
stdout, process.stdout, sys.stdout.buffer),
asyncio.Task(process.stderr.readline()): (
stderr, process.stderr, sys.stderr.buffer)}
while tasks:
done, pending = yield from asyncio.wait(tasks,
return_when=asyncio.FIRST_COMPLETED)
assert done
for future in done:
buf, stream, display = tasks.pop(future)
line = future.result()
if line: # not EOF
buf.append(line) # save for later
display.write(line) # display in terminal
# schedule to read the next line
tasks[asyncio.Task(stream.readline())] = buf, stream, display
# wait for the process to exit
rc = yield from process.wait()
return rc, b''.join(stdout), b''.join(stderr)
脚本运行 '/path/to/script
命令并同时逐行读取其标准输出和标准错误.这些行相应地打印到父级的 stdout/stderr 并保存为字节串以供将来处理.要运行 read_and_display()
协程,我们需要一个事件循环:
The script runs '/path/to/script
command and reads line by line both its stdout&stderr concurrently. The lines are printed to parent's stdout/stderr correspondingly and saved as bytestrings for future processing. To run the read_and_display()
coroutine, we need an event loop:
import os
if os.name == 'nt':
loop = asyncio.ProactorEventLoop() # for subprocess' pipes on Windows
asyncio.set_event_loop(loop)
else:
loop = asyncio.get_event_loop()
try:
rc, *output = loop.run_until_complete(read_and_display("/path/to/script"))
if rc:
sys.exit("child failed with '{}' exit code".format(rc))
finally:
loop.close()
相关文章