问题描述

来自

高级算法:

正如@lunixbochs 提出的答案，sys._getframe() 基本上是 C 代码中的 stackdepth1().虽然更简单的算法总是从堆栈顶部的现有帧开始在 Python 中进行深度搜索，向下检查堆栈以查找更多现有帧，stacksize4b() 允许开始通过其 stack_hint 参数从任何级别搜索，如果需要，可以向下或向上搜索堆栈.

在底层，调用 sys._getframe() 总是意味着将堆栈从顶部框架向下移动到指定深度.因为 Python 和 C 之间的性能差异如此之大，如果需要在应用基本关闭之前找到更接近最深的帧，多次调用 sys._getframe() 仍然可以得到回报- 在 Python 中使用 frame.f_back 进行逐帧搜索.

从 itertools 导入计数def stack_size4b(size_hint=8):"""获取调用者帧的堆栈大小."""get_frame = sys._getframe框架 = 无尝试:而真:框架 = get_frame(size_hint)大小提示 *= 2除了ValueError:如果框架:size_hint//= 2别的:虽然不是框架:size_hint = max(2, size_hint//2)尝试:框架 = get_frame(size_hint)除了ValueError:继续计数中的大小(size_hint):框架 = frame.f_back如果不是框架:返回大小

stacksize4b() 的使用理念是将大小提示放置在您预期的堆栈深度的下限处，以便快速开始，同时仍然能够应对每一次剧烈和短暂的- 堆栈深度的实时变化.

基准测试显示 stacksize4b() 与默认 size_hint=8 和调整后的 size_hint=200.对于基准测试，3-3000 范围内的所有堆栈深度都经过测试，以显示 stacksize4b() 时序中的特征锯齿模式.

From the documentation:

sys.getrecursionlimit()

Return the current value of the recursion limit, the maximum depth of the Python interpreter stack. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python. It can be set by setrecursionlimit().

I am currently hitting the recursion limit when pickling an object. The object I am pickling only has a few levels of nesting, so I am a bit puzzled by what is happening.

I have been able to circumvent the issue with the following hack:

try:
    return pickle.dumps(x)
except:
    try:
        recursionlimit = getrecursionlimit()
        setrecursionlimit(2*recursionlimit)
        dumped = pickle.dumps(x)
        setrecursionlimit(recursionlimit)
        return dumped
    except:
        raise

Testing the above snippet on different contexts sometimes leads to success on the first try, and sometimes it leads to success on the second try. So far I have not been able to make it raise the exception.

To further debug my issue it would be helpful to have a way to obtain the current depth of the stack. That would allow me to verify if the entering stack depth is determining whether the snippet above will succeed on the first try or on the second.

Does the standard library provide a function to get the depth of the stack, or if not, how can I obtain it?

def get_stack_depth():
    # what goes here?

解决方案

If speed is an issue, it's way faster to bypass inspect module.

testing depth: 50 (CPython 3.7.3)
stacksize4b()         | depth: 50   |    2.0 µs
stacksize4b(200)      | depth: 50   |    2.2 µs
stacksize3a()         | depth: 50   |    2.4 µs
stacksize2a()         | depth: 50   |    2.9 µs
stackdepth2()         | depth: 50   |    3.0 µs
stackdepth1()         | depth: 50   |    3.0 µs
stackdepth3()         | depth: 50   |    3.4 µs
stacksize1()          | depth: 50   |    7.4 µs  # deprecated
len(inspect.stack())  | depth: 50   |    1.9 ms

I shortened the name of my functions to stacksize() and for easier differentiation, I'm referring to @lunixbochs' functions as stackdepth().

---

Basic Algorithms:

That's probably the best compromise between code brevity, readability and speed for small stack sizes. For under ~10 frames, only stackdepth1() is slightly faster due to lower overhead.

from itertools import count

def stack_size2a(size=2):
    """Get stack size for caller's frame.
    """
    frame = sys._getframe(size)

    for size in count(size):
        frame = frame.f_back
        if not frame:
            return size

For achieving better timings for larger stack sizes, some more refined algorithms are possible. stacksize3a() is combining chained attribute lookup with a close range finish from stackdepth1() for a much more favorable slope in timings, starting to pay off for roughly > 70 frames in my benchmarks.

from itertools import count

def stack_size3a(size=2):
    """Get stack size for caller's frame.
    """
    frame = sys._getframe(size)
    try:
        for size in count(size, 8):
            frame = frame.f_back.f_back.f_back.f_back.
                f_back.f_back.f_back.f_back
    except AttributeError:
        while frame:
            frame = frame.f_back
            size += 1
        return size - 1

Advanced Algorithms:

As @lunixbochs has brought up in an answer, sys._getframe() is basically stackdepth1() in C-code. While simpler algorithms always start their depth-search in Python from an existing frame at the top of stack, checking the stack downward for further existing frames, stacksize4b() allows starting the search from any level by its stack_hint-parameter and can search the stack down- or upward if needed.

Under the hood, calling sys._getframe() always means walking the stack from the top frame downward to a specified depth. Because the performance difference between Python and C is so huge, it can still pay off to call sys._getframe() multiple times if necessary to find a frame closer to the deepest one, before applying a basic close-range frame-by-frame search in Python with frame.f_back.

from itertools import count

def stack_size4b(size_hint=8):
    """Get stack size for caller's frame.
    """
    get_frame = sys._getframe
    frame = None
    try:
        while True:
            frame = get_frame(size_hint)
            size_hint *= 2
    except ValueError:
        if frame:
            size_hint //= 2
        else:
            while not frame:
                size_hint = max(2, size_hint // 2)
                try:
                    frame = get_frame(size_hint)
                except ValueError:
                    continue

    for size in count(size_hint):
        frame = frame.f_back
        if not frame:
            return size

The usage-idea of stacksize4b() is to place the size-hint at the lower bound of your expected stack depth for a jump start, while still being able to cope with every drastic and short-lived change in stack-depth.

The benchmark shows stacksize4b() with default size_hint=8 and adjusted size_hint=200. For the benchmark all stack depths in the range 3-3000 have been tested to show the characteristic saw pattern in timings for stacksize4b().