Python多处理简单的方法来实现一个简单的计数器?

2022-01-12 00:00:00 python multiprocessing

问题描述

大家好，我现在在 python 中使用多处理.我只是想知道是否存在某种简单的计数器变量，每个进程在完成处理某些任务时可以增加(有点像总共完成了多少工作).

Hey everyone, I am using multiprocessing in python now. and I am just wondering whether there exists some sort of simple counter variable that each process when they are done processing some task could just increment ( kind of like how much work done in total).

我查找了 Value 的 API，不认为它是可变的.

I looked up the API for Value, don't think it's mutable.

解决方案

Value 确实是可变的；您从 ctypes 模块中指定所需的数据类型，然后可以对其进行变异.这是一个完整的工作脚本来演示这一点:

Value is indeed mutable; you specify the datatype you want from the ctypes module and then it can be mutated. Here's a complete, working script that demonstrates this:

from time import sleep
from ctypes import c_int
from multiprocessing import Value, Lock, Process

counter = Value(c_int)  # defaults to 0
counter_lock = Lock()
def increment():
    with counter_lock:
        counter.value += 1

def do_something():
    print("I'm a separate process!")
    increment()

Process(target=do_something).start()
sleep(1)
print counter.value   # prints 1, because Value is shared and mutable

Luper 在下面的评论中正确指出 Value 值默认被锁定.这是正确的，即使一个赋值由多个操作组成(例如赋值一个可能有很多字符的字符串)，那么这个赋值也是原子的.但是，当递增计数器时，您仍然需要一个外部锁，如我的示例中提供的那样，因为递增加载当前值，然后递增它，然后将结果分配回 Value.

Luper correctly points out in a comment below that Value values are locked by default. This is correct in the sense that even if an assignment consists of multiple operations (such as assigning a string which might be many characters) then this assignment is atomic. However, when incrementing a counter you'll still need an external lock as provided in my example, because incrementing loads the current value and then increments it and then assigns the result back to the Value.

所以如果没有外部锁，你可能会遇到以下情况:

So without an external lock, you might run into the following circumstance:

  • 进程 1(以原子方式)读取计数器的当前值，然后将其递增
  • 在进程 1 可以将递增的计数器分配回 Value 之前，会发生上下文切换
  • 进程 2 (原子地)读取计数器的当前(未递增)值，递增它，并将递增的结果(原子地)分配回 Value
  • 进程 1 分配其增量值(原子地)，消除进程 2 执行的增量
  • Process 1 reads (atomically) the current value of the counter, then increments it
  • before Process 1 can assign the incremented counter back to the Value, a context switch occurrs
  • Process 2 reads (atomically) the current (unincremented) value of the counter, increments it, and assigns the incremented result (atomically) back to Value
  • Process 1 assigns its incremented value (atomically), blowing away the increment performed by Process 2

相关文章