boost线程抛出异常“thread_resource_error:资源暂时不可用"

2022-01-08 00:00:00 multithreading exception resources c++ boost

我有类似下面的代码

boost::thread myThread
unsigned char readbuffer[bignumber];
unsigned char writebuffer[bignumber];

for(int i=0; i<bignumber; ++i){
  functiondostuff();
  for(int j=0; j<2; ++j){
    functiondomorestuff();
    myThread = boost::thread(&myClass::myFunction, this, j, i);
  }     
}

myFunction 从缓冲区读取并写入另一个.它永远不会写入写入缓冲区中的相同位置.我在这里对线程做一些根本错误的事情吗?循环使用相同的线程名称创建线程是不是很糟糕?它运行了一段时间，然后我得到以下异常.

myFunction reads from a buffer and writes to another. It will never write to the same location in the write buffer. Am I doing something fundamentally wrong with threads here? Is it bad to loop over a thread creation with the same thread name? It runs smooth for a while and then I get the following exception.

在抛出 'boost::exception_detail::clone_impl >' 的实例后调用终止what(): boost::thread_resource_error: 资源暂时不可用中止

terminate called after throwing an instance of 'boost::exception_detail::clone_impl >' what(): boost::thread_resource_error: Resource temporarily unavailable Aborted

这个异常是什么意思?任何想法都会有所帮助.

What does this exception mean? Any ideas would be helpful.

推荐答案

每个进程可以创建的线程数是有限制的.

There's a limit on the number of threads you can create per process.

以linux为例，

cat /proc/sys/kernel/threads-max

告诉你当前的最大值.默认值是内存页数/4，所以在我的系统上它是 513785，但在另一个盒子上它可能要低得多.例如.在我的邮件服务器箱(512mb RAM)上它只有 7295.

tells you the current maximum. The default is the number of memory pages/4, so on my system it's 513785, but it may be much much lower on another box. E.g. on my mail server box (512mb RAM) it's only 7295.

你可以限制.但实际上这将毫无用处，因为操作系统无法有效地安排它们.因此，请尝试使用线程池.

You could the limit. But in fact that will be useless because the OS can't schedule them effectively. So, instead, try using a thread pool.

哦.PS. detach()-他的线程将帮助(很多)节约资源.pthreads 可能会在达到操作系统限制之前阻止线程创建，因为它需要分配跟踪活动线程的开销.detach 将它们释放(并消除程序退出前未加入所有线程的错误).

Oh. PS. detach()-ing he threads will help (a lot) with conserving resources. pthreads might be blocking thread creation well before the OS limit is reached because it needs to allocate overhead tracking the active threads. detach frees those up (and removes the error of not joining all threads before program exit).

更新疯狂的星期五奖励:一个线程池，可自动缩放到您的系统拥有的核心数量:

UPDATE Crazy friday bonus: a thread pool that auto-scales to the number of cores your system has:

#include <boost/thread.hpp>
#include <boost/phoenix.hpp>
#include <boost/optional.hpp>

using namespace boost;
using namespace boost::phoenix::arg_names;

boost::atomic_size_t counter(0ul);

class thread_pool
{
  private:
      mutex mx;
      condition_variable cv;

      typedef function<void()> job_t;
      std::deque<job_t> _queue;

      thread_group pool;

      boost::atomic_bool shutdown;
      static void worker_thread(thread_pool& q)
      {
          while (auto job = q.dequeue())
              (*job)();
      }

  public:
      thread_pool() : shutdown(false) {
          for (unsigned i = 0; i < boost::thread::hardware_concurrency(); ++i)
              pool.create_thread(bind(worker_thread, ref(*this)));
      }

      void enqueue(job_t job) 
      {
          lock_guard<mutex> lk(mx);
          _queue.push_back(std::move(job));

          cv.notify_one();
      }

      optional<job_t> dequeue() 
      {
          unique_lock<mutex> lk(mx);
          namespace phx = boost::phoenix;

          cv.wait(lk, phx::ref(shutdown) || !phx::empty(phx::ref(_queue)));

          if (_queue.empty())
              return none;

          auto job = std::move(_queue.front());
          _queue.pop_front();

          return std::move(job);
      }

      ~thread_pool()
      {
          shutdown = true;
          {
              lock_guard<mutex> lk(mx);
              cv.notify_all();
          }

          pool.join_all();
      }
};

static constexpr size_t bignumber = 1 << 20;

class myClass 
{
    //unsigned char readbuffer[bignumber];
    //unsigned char writebuffer[bignumber];
    void functiondostuff() { }
    void functiondomorestuff() { }

    thread_pool pool; // uses 1 thread per core

  public:
    void wreak_havoc()
    {
        std::cout << "enqueuing jobs... " << std::flush;
        for(size_t i=0; i<bignumber; ++i)
        {
            functiondostuff();
            for(int j=0; j<2; ++j) {
                functiondomorestuff();
                pool.enqueue(bind(&myClass::myFunction, this, j, i));
            }     
        }
        std::cout << "done
";
    }

  private:
    void myFunction(int i, int j)
    {
        boost::this_thread::sleep_for(boost::chrono::milliseconds(1));
        counter += 1;
    }
};

int main()
{
    myClass instance;
    instance.wreak_havoc();

    size_t last = 0;
    while (counter < (2*bignumber))
    {
        boost::this_thread::sleep_for(boost::chrono::milliseconds(100));
        if ((counter >> 4u) > last)
        {
            std::cout << "Progress: " << counter << "/" << (bignumber*2) << "
";
            last = counter >> 4u;
        }
    }
}

相关文章