迭代二进制文件的惯用方法是什么?

2022-01-10 00:00:00 python file iterator

问题描述

使用文本文件，我可以这样写:

With a text file, I can write this:

with open(path, 'r') as file:
    for line in file:
        # handle the line

这等价于:

with open(path, 'r') as file:
    for line in iter(file.readline, ''):
        # handle the line

这个成语记录在 PEP 234 但我没有找到二进制文件的类似习语.

This idiom is documented in PEP 234 but I have failed to locate a similar idiom for binary files.

使用二进制文件，我可以这样写:

With a binary file, I can write this:

with open(path, 'rb') as file:
    while True:
        chunk = file.read(1024 * 64)
        if not chunk:
            break
        # handle the chunk

我尝试过与文本文件相同的习语:

I have tried the same idiom that with a text file:

def make_read(file, size):
    def read():
        return file.read(size)
    return read

with open(path, 'rb') as file:
    for chunk in iter(make_read(file, 1024 * 64), b''):
        # handle the chunk

这是在 Python 中迭代二进制文件的惯用方式吗?

Is it the idiomatic way to iterate over a binary file in Python?

解决方案

我不知道有什么内置的方法可以做到这一点，但是一个包装函数很容易写:

I don't know of any built-in way to do this, but a wrapper function is easy enough to write:

def read_in_chunks(infile, chunk_size=1024*64):
    while True:
        chunk = infile.read(chunk_size)
        if chunk:
            yield chunk
        else:
            # The chunk was empty, which means we're at the end
            # of the file
            return

然后在交互式提示处:

>>> from chunks import read_in_chunks
>>> infile = open('quicklisp.lisp')
>>> for chunk in read_in_chunks(infile):
...     print chunk
... 
<contents of quicklisp.lisp in chunks>

当然，您可以轻松地将其调整为使用 with 块:

Of course, you can easily adapt this to use a with block:

with open('quicklisp.lisp') as infile:
    for chunk in read_in_chunks(infile):
        print chunk

你可以像这样去掉 if 语句.

And you can eliminate the if statement like this.

def read_in_chunks(infile, chunk_size=1024*64):
    chunk = infile.read(chunk_size)
    while chunk:
        yield chunk
        chunk = infile.read(chunk_size)

相关文章