将数据从 glReadPixels() 转换为 OpenCV::Mat

2021-12-10 00:00:00 opencv opengl c++ glreadpixels

我想使用 glReadPixels() 从动画中获取每个 OpenGL 帧，并将数据转换为 OpenCV::Mat.我知道 glReadPixels() 从下到上，从左到右按行获取数据.另一方面，OpenCV 以不同的方式存储数据.

I want to get every OpenGL frame from an animation with glReadPixels() and convert the data to OpenCV::Mat. I know that glReadPixels() gets the data by rows from the lower one to upper one, from left to right. On the other hand, OpenCV stores the data differently.

有没有人知道任何可以帮助我将数据从 glReadPixels 转换为 C++ 中的 OpenCV:Mat 的库或任何教程/示例?

Does anybody know any library or any tutorial/example that helps me to convert data from glReadPixels to a OpenCV:Mat in C++?

总结

 OpenGL frame      ----------------------->        CV::Mat

Data from left to right,                    Data from left to right,
bottom to top.                              top to bottom.

推荐答案

首先我们创建一个空的(或单元化的)cv::Mat 以便我们的数据可以直接读入.这可以在启动时完成一次，但另一方面 cv::Mat::create 当图像已经具有匹配的大小和类型时并不会真正花费太多.类型取决于您的需求，通常类似于 CV_8UC3 用于 24 位彩色图像.

First we create an empty (or unititialized) cv::Mat for our data to be read into directly. This can be done once at startup, but on the other hand cv::Mat::create doesn't really cost much when the image already has matching size and type. The type depends on your needs, usually it's something like CV_8UC3 for a 24-bit color image.

cv::Mat img(height, width, CV_8UC3);

或

img.create(height, width, CV_8UC3);

然后您必须考虑 cv::Mat 不必连续存储图像行.每行末尾可能有一个小的填充值，以使行 4 字节对齐(或 8 字节?).所以你需要弄乱像素存储模式:

Then you have to account for cv::Mat not neccessarily storing image rows contiguously. There might be a small padding value at the end of each row to make rows 4-byte aligned (or 8?). So you need to mess with the pixel storage modes:

//use fast 4-byte alignment (default anyway) if possible
glPixelStorei(GL_PACK_ALIGNMENT, (img.step & 3) ? 1 : 4);

//set length of one complete row in destination data (doesn't need to equal img.cols)
glPixelStorei(GL_PACK_ROW_LENGTH, img.step/img.elemSize());

接下来，矩阵的类型会影响glReadPixels 的格式和类型参数.如果你想要彩色图像，你必须记住 OpenCV 通常以 BGR 顺序存储颜色值，所以你需要使用 GL_BGR(A)(这是在 OpenGL 1.2 中添加的)而不是 GL_RGB(A).对于一个分量图像，使用 GL_LUMINANCE(对各个颜色分量求和)或 GL_RED、GL_GREEN、...(获取单个分量).因此，对于我们的 CV_8UC3 图像，将其直接读入 cv::Mat 的最终调用将是:

Next, the type of the matrix influences the format and type parameters of glReadPixels. If you want color images you have to keep in mind that OpenCV usually stores color values in BGR order, so you need to use GL_BGR(A) (which were added with OpenGL 1.2) instead of GL_RGB(A). For one component images use either GL_LUMINANCE (which sums the individual color components) or GL_RED, GL_GREEN, ... (to get an individual component). So for our CV_8UC3 image the final call to read it directly into the cv::Mat would be:

glReadPixels(0, 0, img.cols, img.rows, GL_BGR, GL_UNSIGNED_BYTE, img.data);

最后，OpenCV 从上到下存储图像.因此，您可能需要在获取它们后翻转它们，或者首先在 OpenGL 中渲染它们(这可以通过调整投影矩阵来完成，但在这种情况下要注意三角形方向).要垂直翻转 cv::Mat，您可以使用 cv::flip:

Finally, OpenCV stores images from top to bottom. So you may need to either flip them after getting them or render them flipped in OpenGL in the first place (this can be done by adjusting the projection matrix, but keep an eye on triangle orientation in this case). To flip a cv::Mat vertically, you can use cv::flip:

cv::flip(img, flipped, 0);

所以要记住 OpenCV:

So to keep in mind OpenCV:

  • 从上到下，从左到右存储图像
  • 按 BGR 顺序存储彩色图像
  • 可能不会将图像行存储得紧凑

相关文章