Python 多进程分析

2022-01-12 00:00:00 python multiprocessing cprofile

问题描述

我正在努力弄清楚如何分析一个简单的多进程 python 脚本

I'm struggling to figure out how to profile a simple multiprocess python script

import multiprocessing
import cProfile
import time
def worker(num):
    time.sleep(3)
    print 'Worker:', num

if __name__ == '__main__':
    for i in range(5):
        p = multiprocessing.Process(target=worker, args=(i,))
        cProfile.run('p.start()', 'prof%d.prof' %i)

我正在启动 5 个进程，因此 cProfile 会生成 5 个不同的文件.在每个内部，我想看到我的方法worker"运行大约需要 3 秒，但我只看到start"方法内部发生的事情.

I'm starting 5 processes and therefore cProfile generates 5 different files. Inside of each I want to see that my method 'worker' takes approximately 3 seconds to run but instead I'm seeing only what's going on inside the 'start'method.

如果有人可以向我解释这一点，我将不胜感激.

I would greatly appreciate if somebody could explain this to me.

import multiprocessing
import cProfile
import time
def test(num):
    time.sleep(3)
    print 'Worker:', num

def worker(num):
    cProfile.runctx('test(num)', globals(), locals(), 'prof%d.prof' %num)


if __name__ == '__main__':
    for i in range(5):
        p = multiprocessing.Process(target=worker, args=(i,))
        p.start()

解决方案

你正在分析进程启动，这就是为什么你只看到 p.start() 中发生了什么比如说，一旦子进程启动，p.start() 就会返回.您需要在 worker 方法中进行概要分析，该方法将在子进程中被调用.

You're profiling the process startup, which is why you're only seeing what happens in p.start() as you say—and p.start() returns once the subprocess is kicked off. You need to profile inside the worker method, which will get called in the subprocesses.

相关文章