如何在 Python3 中检测 concurrent.futures 中的异常?

2022-01-12 00:00:00 python python-2.7 python-3.4 concurrent.futures multiprocessing

问题描述

由于它的并发期货模块，我刚刚转向 python3.我想知道是否可以让它检测错误.我想使用并发期货来并行程序，如果有更高效的模块请告诉我.

I have just moved on to python3 as a result of its concurrent futures module. I was wondering if I could get it to detect errors. I want to use concurrent futures to parallel program, if there are more efficient modules please let me know.

我不喜欢多处理，因为它太复杂而且没有多少文档可用.但是，如果有人可以编写一个没有类的 Hello World，只使用多处理并行计算的函数，这样它就很容易理解了，那就太好了.

I do not like multiprocessing as it is too complicated and not much documentation is out. It would be great however if someone could write a Hello World without classes only functions using multiprocessing to parallel compute so that it is easy to understand.

这是一个简单的脚本:

from concurrent.futures import ThreadPoolExecutor

def pri():
    print("Hello World!!!")

def start():
    try:
        while True:
            pri()
    except KeyBoardInterrupt:
        print("YOU PRESSED CTRL+C")


with ThreadPoolExecutor(max_workers=3) as exe:
    exe.submit(start)

以上代码只是一个演示，说明 CTRL+C 如何无法打印语句.

The above code was just a demo, of how CTRL+C will not work to print the statement.

我想要的是能够调用函数是存在错误.这种错误检测必须来自函数本身.

What I want is to be able to call a function is an error is present. This error detection must be from the function itself.

另一个例子

import socket
from concurrent.futures import ThreadPoolExecutor 
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
def con():
    try:
        s.connect((x,y))
        main()
    except: socket.gaierror
         err()
def err():
    time.sleep(1)
    con()
def main():
    s.send("[+] Hello")
with ThreadPoolExecutor as exe:
    exe.submit(con)

解决方案

这里的解决方案.我不确定你是否喜欢它，但我想不出其他的.我已经修改了您的代码以使其正常工作.

Here's a solution. I'm not sure you like it, but I can't think of any other. I've modified your code to make it work.

from concurrent.futures import ThreadPoolExecutor
import time

quit = False

def pri():
    print("Hello World!!!")

def start():
    while quit is not True:
        time.sleep(1)
        pri()

try:
    pool = ThreadPoolExecutor(max_workers=3)
    pool.submit(start)

    while quit is not True:
        print("hei")
        time.sleep(1)
except KeyboardInterrupt:
    quit = True

以下是要点:

  1. 当您使用 with ThreadPoolExecutor(max_workers=3) as exe 时，它会等待所有任务完成.看看 Doc

  1. When you use with ThreadPoolExecutor(max_workers=3) as exe, it waits until all tasks have been done. Have a look at Doc

如果 wait 为 True，则此方法将不会返回，直到所有未决的期货都执行完毕并且与执行程序关联的资源已被释放.如果 wait 为 False，则此方法将立即返回，并且当所有未决的期货执行完毕后，与执行程序关联的资源将被释放.无论 wait 的值如何，整个 Python 程序都不会退出，直到所有未决的期货都执行完毕.

If wait is True then this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed. If wait is False then this method will return immediately and the resources associated with the executor will be freed when all pending futures are done executing. Regardless of the value of wait, the entire Python program will not exit until all pending futures are done executing.

如果您使用 with 语句，您可以避免显式调用此方法，该语句将关闭 Executor(就像 Executor.shutdown() 一样等待 被调用，等待设置为 True)

You can avoid having to call this method explicitly if you use the with statement, which will shutdown the Executor (waiting as if Executor.shutdown() were called with wait set to True)

这就像在线程上调用 join().
这就是为什么我将其替换为:

It's like calling join() on a thread.
That's why I replaced it with: 

pool = ThreadPoolExecutor(max_workers=3)
pool.submit(start)

  • 主线程必须在做工作"才能捕捉到 Ctrl+C.所以你不能把主线程放在那里然后退出，最简单的方法是运行一个无限循环

  • Main thread must be doing "work" to be able to catch a Ctrl+C. So you can't just leave main thread there and exit, the simplest way is to run an infinite loop

    现在你已经在主线程中运行了一个循环，当你按下 CTRL+C 时，程序将进入 except KeyboardInterrupt 块并设置 退出=真.然后你的工作线程就可以退出了.

    Now that you have a loop running in main thread, when you hit CTRL+C, program will enter the except KeyboardInterrupt block and set quit=True. Then your worker thread can exit.

    严格来说，这只是一种解决方法.在我看来，这不可能有其他方法.

    Strictly speaking, this is only a workaround. It seems to me it's impossible to have another way for this.

    编辑
    我不确定是什么在困扰您，但您可以毫无问题地在另一个线程中捕获异常:

    Edit
    I'm not sure what's bothering you, but you can catch exception in another thread without problem:

    import socket
import time
from concurrent.futures import ThreadPoolExecutor 
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)

def con():
    try:
        raise socket.gaierror
        main()
    except socket.gaierror:
        print("gaierror occurred")
        err()

def err():
    print("err invoked")
    time.sleep(1)
    con()

def main():
    s.send("[+] Hello")

with ThreadPoolExecutor(3) as exe:
    exe.submit(con)

    输出

    gaierror occurred
err invoked
gaierror occurred
err invoked
gaierror occurred
err invoked
gaierror occurred
...

    • 相关文章