boost.python 不支持并行?
我正在尝试使用 boost.python 将一段 C++ 代码包装到 python lib 中，但是，我发现多个实例不能同时运行:
I am trying to wrap a piece of C++ code into python lib using boost.python, however, I found out that multiple instances cannot run at the same time:
代码(C++):
class Foo{
public:
Foo(){}
void run(){
int seconds = 2;
clock_t endwait;
endwait = clock () + seconds * CLOCKS_PER_SEC ;
while (clock() < endwait) {}
}
};
BOOST_PYTHON_MODULE(run_test)
{
using namespace boost::python;
class_<Foo>("test", init<>())
.def("run", &Foo::run)
;
}
这是使用 CMake (CMake) 编译的:
which is compile using CMake (CMake):
add_library(run_test SHARED run_test.cpp)
target_link_libraries(run_test boost_python python2.7)
并使用以下代码(Python)进行测试:
and tested with the following code (Python):
class Dos(threading.Thread):
def run(self):
printl('performing DoS attack')
proc = test()
proc.run()
for i in range(5):
t = Dos()
t.start()
输出表明代码以一种非常奇怪的方式并行化.每个线程应该只需要 2 秒，并且 4 个线程应该在我的四核机器上同时运行:
The output indicates that the code is parallelized in a very weird way. Each thread should take only 2 seconds and 4 threads should run simultaneously on my quadcore machine:
[2011-11-04 13:57:01] performing DoS attack
[2011-11-04 13:57:01] performing DoS attack
[2011-11-04 13:57:05] performing DoS attack
[2011-11-04 13:57:05] performing DoS attack
[2011-11-04 13:57:09] performing DoS attack
感谢您的帮助！
推荐答案
你遇到的是 python Global Interpreter Lock.GIL 一次只允许一个线程在 python 解释器中运行.
What you are running into is the python Global Interpreter Lock. The GIL only allows one thread at a time to run in the python interpreter.
Boost.Python 的一个优点是你可以释放 GIL，做 C++ 的东西，然后在你完成后收回.但这也是一种责任.Python 通常会定期释放 GIL，以便让其他线程有机会运行.如果您使用 C++，这就是您的工作.如果您在持有 GIL 的同时处理数字 2 小时，您将冻结整个口译员.
One of the advantages of Boost.Python is that you can release the GIL, do C++ stuff, and then take it back when you are done. This is also a responsibility however. Python normally releases the GIL at regular intervals, to give other threads a chance to run. If you are in C++, this is your job. If you go crunch numbers for 2 hours while holding the GIL, you will freeze the whole interpreter.
这可以通过一点点反向 RAII 轻松解决:
This can be easy to fix with a little reverse RAII:
class releaseGIL{
public:
inline releaseGIL(){
save_state = PyEval_SaveThread();
}
inline ~releaseGIL(){
PyEval_RestoreThread(save_state);
}
private:
PyThreadState *save_state;
};
现在您可以像这样更改代码:
Now you can change your code like so:
class Foo{
public:
Foo(){}
void run(){
{
releaseGIL unlock = releaseGIL();
int seconds = 2;
clock_t endwait;
endwait = clock () + seconds * CLOCKS_PER_SEC ;
while (clock() < endwait) {}
}
}
};
请务必注意，在不持有 GIL 的情况下，您不得触摸任何 Python 代码或 Python 数据或调用解释器.这将导致您的解释器崩溃.
It is VERY important to note that you MUST NOT touch any python code, or python data or call in to the interpreter while not holding the GIL. This will cause your interpreter to crash.
也可以走另一条路.当前不持有 GIL 的线程可以获取它，并调用 python.这可以是较早发布 GIL 的线程，也可以是使用 C++ 启动但从未拥有 GIL 的线程.这是 RAII 类:
It is also possible to go the other way. A thread not currently holding the GIL can acquire it, and make calls in to python. This can be a thread that released the GIL earlier, or one that started in c++ and never had the GIL. Here is the RAII class for that:
class AcquireGIL
{
public:
inline AcquireGIL(){
state = PyGILState_Ensure();
}
inline ~AcquireGIL(){
PyGILState_Release(state);
}
private:
PyGILState_STATE state;
};
用法留给学生作为练习.
Usage is left as an exercise for the student.
附加说明(我总是忘记提及这一点):
Additional note (I always forget to mention this):
如果您打算在 C++ 中使用 GIL，则您的模块定义需要从以下代码开始:
If you are going to be messing with the GIL in c++ your module definition needs to start with this code:
BOOST_PYTHON_MODULE(ModuleName)
{
PyEval_InitThreads();
...
}
相关文章