由于捕获缓冲区，OpenCV VideoCapture 滞后

2021-12-10 00:00:00 opencv c++ video

我正在通过提供 mjpeg 流的网络摄像头捕获视频.我在工作线程中进行了视频捕获.我这样开始捕获:

I am capturing video through a webcam which gives a mjpeg stream. I did the video capture in a worker thread. I start the capture like this:

const std::string videoStreamAddress = "http://192.168.1.173:80/live/0/mjpeg.jpg?x.mjpeg";
qDebug() << "start";
cap.open(videoStreamAddress);
qDebug() << "really started";
cap.set(CV_CAP_PROP_FRAME_WIDTH, 720);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 576);

相机以 20fps 的速度馈送视频流.但是如果我像这样以 20fps 的速度阅读:

the camera is feeding the stream at 20fps. But if I did the reading in 20fps like this:

if (!cap.isOpened()) return;

        Mat frame;
        cap >> frame; // get a new frame from camera
        mutex.lock();

        m_imageFrame = frame;
        mutex.unlock();

然后有 3 秒以上的延迟.原因是采集到的视频首先存放在一个缓冲区中.当我第一次启动相机时，缓冲区是累积的，但我没有把帧读出来.所以如果我从缓冲区读取它总是给我旧的帧.我现在唯一的解决方案是以 30fps 读取缓冲区，这样它就会快速清理缓冲区，并且不会出现更严重的延迟.

Then there is a 3+ seconds lag. The reason is that the captured video is first stored in a buffer.When I first start the camera, the buffer is accumulated but I did not read the frames out. So If I read from the buffer it always gives me the old frames. The only solutions I have now is to read the buffer at 30fps so it will clean the buffer quickly and there's no more serious lag.

有没有其他可能的解决方案，以便我每次启动相机时都可以手动清理/刷新缓冲区?

Is there any other possible solution so that I could clean/flush the buffer manually each time I start the camera?

推荐答案

OpenCV 解决方案

根据这个源，您可以设置cv::VideoCapture 对象的缓冲区大小.

OpenCV Solution

According to this source, you can set the buffersize of a cv::VideoCapture object.

cv::VideoCapture cap;
cap.set(CV_CAP_PROP_BUFFERSIZE, 3); // internal buffer will now store only 3 frames

// rest of your code...

但是有一个重要的限制:

There is an important limitation however:

CV_CAP_PROP_BUFFERSIZE 存储在内部缓冲存储器中的帧数(注意:目前仅支持 DC1394 v 2.x 后端)

CV_CAP_PROP_BUFFERSIZE Amount of frames stored in internal buffer memory (note: only supported by DC1394 v 2.x backend currently)

从评论中更新.在较新版本的 OpenCV (3.4+) 中，限制似乎消失了，代码使用了作用域枚举:

Update from comments. In newer versions of OpenCV (3.4+), the limitation seems to be gone and the code uses scoped enumerations:

cv::VideoCapture cap;
cap.set(cv::CAP_PROP_BUFFERSIZE, 3);

<小时>

解决方法 1

如果解决方案不起作用，请查看这篇博文 解释了如何解决这个问题.

Hackaround 1

If the solution does not work, take a look at this post that explains how to hack around the issue.

简而言之:测量查询一帧所需的时间；如果它太低，则表示该帧是从缓冲区中读取的，可以丢弃.继续查询帧，直到测量的时间超过某个限制.发生这种情况时，缓冲区为空，返回的帧是最新的.

In a nutshell: the time needed to query a frame is measured; if it is too low, it means the frame was read from the buffer and can be discarded. Continue querying frames until the time measured exceeds a certain limit. When this happens, the buffer was empty and the returned frame is up to date.

(链接帖子上的答案显示:从缓冲区返回帧的时间大约是返回最新帧的时间的 1/8.当然，您的里程可能会有所不同！)

(The answer on the linked post shows: returning a frame from the buffer takes about 1/8th the time of returning an up to date frame. Your mileage may vary, of course!)

一个不同的解决方案，灵感来自这篇帖子, 是创建第三个线程，高速连续抓取帧，保持缓冲区为空.这个线程应该使用 cv::VideoCapture.grab() 以避免开销.

A different solution, inspired by this post, is to create a third thread that grabs frames continuously at high speed to keep the buffer empty. This thread should use the cv::VideoCapture.grab() to avoid overhead.

您可以使用一个简单的自旋锁来同步真正的工作线程和第三个线程之间的阅读帧.

You could use a simple spin-lock to synchronize reading frames between the real worker thread and the third thread.

相关文章