子进程Popen和通信后关闭所有文件的正确方法
问题描述
我们在运行 python Twisted 应用程序的 Ubuntu Linux 机器上遇到了一些可怕的打开的文件太多"的问题.在我们程序的许多地方，我们都在使用子进程 Popen，如下所示:
We are having some problems with the dreaded "too many open files" on our Ubuntu Linux machine rrunning a python Twisted application. In many places in our program, we are using subprocess Popen, something like this:
Popen('ifconfig ' + iface, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
output = process.stdout.read()
而在其他地方我们使用子进程通信:
while in other places we use subprocess communicate:
process = subprocess.Popen(['/usr/bin/env', 'python', self._get_script_path(script_name)],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
close_fds=True)
out, err = process.communicate(data)
在这两种情况下我究竟需要做什么才能关闭任何打开的文件描述符?Python 文档对此并不清楚.根据我收集的信息(可能是错误的)，communicate() 和 wait() 确实会自行清理任何打开的 fd.但是波本呢?如果我不调用通信或等待，我是否需要在调用 Popen 后显式关闭标准输入、标准输出和标准错误?
What exactly do I need to do in both cases in order to close any open file descriptors? Python documentation is not clear on this. From what I gather (which could be wrong) both communicate() and wait() will indeed clean up any open fds on their own. But what about Popen? Do I need to close stdin, stdout, and stderr explicitly after calling Popen if I don't call communicate or wait?
解决方案
根据到子进程模块的这个源(链接) 如果你调用 communicate 你不应该需要关闭 stdout 和 stderr管道.
According to this source for the subprocess module (link) if you call communicate you should not need to close the stdout and stderr pipes.
否则我会尝试:
process.stdout.close()
process.stderr.close()
在你使用完 process 对象之后.
after you are done using the process object.
例如，当你直接调用 .read() 时:
For instance, when you call .read() directly:
output = process.stdout.read()
process.stdout.close()
查看上面的模块源代码，了解 communicate() 是如何定义的，你会看到它在读取每个管道后会关闭它，所以这也是你应该做的.
Look in the above module source for how communicate() is defined and you'll see that it closes each pipe after it reads from it, so that is what you should also do.
相关文章