我可以在 Python 中将 io.BytesIO() 流通过管道传输到 subprocess.popen() 吗?

2022-01-18 00:00:00 python subprocess popen bytesio pipe

问题描述

我正在尝试使用 subprocess.popen() 将 io.BytesIO() 字节流管道 到一个单独的程序，但我不知道如何或是否有可能.文档和示例都是关于文本和换行符的.

I'm trying to pipe a io.BytesIO() bytetream to a separate program using subprocess.popen(), but I don't know how or if this is at all possible. Documentation and examples are all about text and newlines.

当我做出这样的事情时:

When I whip up something like this:

import io
from subprocess import *

stream = io.BytesIO()
someStreamCreatingProcess(stream)

command = ['somecommand', 'some', 'arguments']  
process = Popen(command, stdin=PIPE)
process.communicate(input=stream)

我明白了

Traceback (most recent call last):
  File "./test.py", line 9, in <module>
    procOut         = process.communicate(input=stream)
  File "/usr/lib/python2.7/subprocess.py", line 754, in communicate
    return self._communicate(input)
  File "/usr/lib/python2.7/subprocess.py", line 1322, in _communicate
    stdout, stderr = self._communicate_with_poll(input)
  File "/usr/lib/python2.7/subprocess.py", line 1384, in _communicate_with_poll
    chunk = input[input_offset : input_offset + _PIPE_BUF]
TypeError: '_io.BytesIO' object has no attribute '__getitem__'

我认为 popen() 仅适用于文本.我错了吗?
有没有其他方法可以做到这一点?

I think popen() is only for text. Am I wrong?
Is there a different way to do this?

解决方案

正如 @falsetru 所说你不能流式传输BytesIO() 对象直接；您需要先从中获取一个字节串.这意味着所有内容都应该已经写入 stream 在 你调用 stream.getvalue() 传递给 process.communicate().

As @falsetru said you can't stream BytesIO() object directly; you need to get a bytestring from it first. It implies that all content should be already written to stream before you call stream.getvalue() to pass to process.communicate().

如果你想 stream 而不是一次提供所有输入，那么你可以删除 BytesIO() 对象并直接写入管道:

If you want to stream instead of providing all input at once then you could drop BytesIO() object and write to the pipe directly:

from subprocess import Popen, PIPE

process = Popen(['command', 'arg1'], stdin=PIPE, bufsize=-1)
someStreamCreatingProcess(stream=process.stdin) # many `stream.write()` inside
process.stdin.close() # done (no more input)
process.wait()

someStreamCreatingProcess() 在完成写入流之前不应返回.如果它立即返回，那么它应该在将来的某个时间调用 stream.close() (删除代码中的 process.stdin.close()):

someStreamCreatingProcess() should not return until it is done writing to the stream. If it returns immediately then it should call stream.close() at some point in the future (remove process.stdin.close() in your code):

from subprocess import Popen, PIPE

process = Popen(['command', 'arg1'], stdin=PIPE, bufsize=-1)
someStreamCreatingProcess(stream=process.stdin) # many `stream.write()` inside
process.wait() # stream.close() is called in `someStreamCreatingProcess`

相关文章