多处理:使用 tqdm 显示进度条

2022-01-12 00:00:00 python multiprocessing tqdm progress-bar

问题描述

为了使我的代码更pythonic"和更快，我使用多处理"和一个映射函数来发送它a)函数和b)迭代范围.

To make my code more "pythonic" and faster, I use "multiprocessing" and a map function to send it a) the function and b) the range of iterations.

植入的解决方案(即直接在范围 tqdm.tqdm(range(0, 30)) 上调用 tqdm)不适用于多处理(如下面的代码所示).

The implanted solution (i.e., call tqdm directly on the range tqdm.tqdm(range(0, 30)) does not work with multiprocessing (as formulated in the code below).

进度条显示从0到100%(python读取代码时?)但并不表示map函数的实际进度.

The progress bar is displayed from 0 to 100% (when python reads the code?) but it does not indicate the actual progress of the map function.

如何显示进度条，指示地图"功能在哪一步?

from multiprocessing import Pool
import tqdm
import time

def _foo(my_number):
   square = my_number * my_number
   time.sleep(1)
   return square 

if __name__ == '__main__':
   p = Pool(2)
   r = p.map(_foo, tqdm.tqdm(range(0, 30)))
   p.close()
   p.join()

欢迎任何帮助或建议...

Any help or suggestions are welcome...

解决方案

找到的解决方案:小心！由于多处理，估计时间(每个循环的迭代次数、总时间等)可能不稳定，但进度条运行良好.

Solution Found : Be careful! Due to multiprocessing, estimation time (iteration per loop, total time, etc.) could be unstable, but the progress bar works perfectly.

注意:Pool 的上下文管理器仅适用于 Python 3.3 版

Note: Context manager for Pool is only available from Python version 3.3

from multiprocessing import Pool
import time
from tqdm import *

def _foo(my_number):
   square = my_number * my_number
   time.sleep(1)
   return square 

if __name__ == '__main__':
    with Pool(processes=2) as p:
        max_ = 30
        with tqdm(total=max_) as pbar:
            for i, _ in enumerate(p.imap_unordered(_foo, range(0, max_))):
                pbar.update()

相关文章