Android:如何通过回调显示相机预览?
我需要做的很简单，我想使用相机回调手动显示来自相机的预览，并且我想在真实设备上获得至少 15fps.我什至不需要颜色，我只需要预览灰度图像.来自相机的图像是 YUV 格式，您必须以某种方式对其进行处理，这是主要的性能问题.我正在使用 API 8.
What I need to do is quite simple, I want to manually display preview from camera using camera callback and I want to get at least 15fps on a real device. I don't even need the colors, I just need to preview grayscale image. Images from camera are in YUV format and you have to process it somehow, which is the main performance problem. I'm using API 8.
在所有情况下，我都使用 camera.setPreviewCallbackWithBuffer()，它比 camera.setPreviewCallback() 快.如果我不显示预览，我似乎无法在这里获得大约 24 fps.所以没有问题.
In all cases I'm using camera.setPreviewCallbackWithBuffer(), that is faster than camera.setPreviewCallback(). It seems that I cant get about 24 fps here, if I'm not displaying the preview. So there is not the problem.
我已经尝试了这些解决方案:
I have tried these solutions:
1.在 SurfaceView 上将相机预览显示为位图. 可以，但性能约为 6fps.
1. Display camera preview on a SurfaceView as a Bitmap. It works, but the performance is about 6fps.
baos = new ByteOutputStream();
yuvimage=new YuvImage(cameraFrame, ImageFormat.NV21, prevX, prevY, null);
yuvimage.compressToJpeg(new Rect(0, 0, prevX, prevY), 80, baos);
jdata = baos.toByteArray();
bmp = BitmapFactory.decodeByteArray(jdata, 0, jdata.length); // Convert to Bitmap, this is the main issue, it takes a lot of time
canvas.drawBitmap(bmp , 0, 0, paint);
<强>2.在 GLSurfaceView 上将相机预览显示为纹理. 这里我只显示亮度数据(灰度图像)，这很简单，每帧只需要一个 arraycopy().我可以获得大约 12fps，但我需要对预览应用一些过滤器，而且似乎在 OpenGL ES 1 中无法快速完成.所以我不能使用这个解决方案.另一个问题中的一些细节.
2. Display camera preview on a GLSurfaceView as a texture. Here I was displaying only luminance data (greyscale image), which is quite easy, it requires only one arraycopy() on each frame. I can get about 12fps, but I need to apply some filters to the preview and it seems, that it can't be done fast in OpenGL ES 1. So I can't use this solution. Some details of this in another question.
3.使用 NDK 在 (GL)SurfaceView 上显示相机预览以处理 YUV 数据.我在 here 找到了一个使用一些 C 函数和 NDK 的解决方案.但我没有设法使用它，这里有更多细节.但无论如何，这个解决方案是返回 ByteBuffer 以在 OpenGL 中将其显示为纹理，并且不会比之前的尝试快.所以我必须修改它以返回 int[] 数组，可以用 canvas.drawBitmap() 绘制，但我对 C 的理解不足以做到这一点.
3. Display camera preview on a (GL)SurfaceView using NDK to process the YUV data. I find a solution here that uses some C function and NDK. But I didn't manage to use it, here some more details. But anyway, this solution is done to return ByteBuffer to display it as a texture in OpenGL and it won't be faster than the previous attempt. So I would have to modify it to return int[] array, that can be drawn with canvas.drawBitmap(), but I don't understand C enough to do this.
那么，我是否还有其他方法遗漏或改进了我尝试的尝试?
So, is there any other way that I'm missing or some improvement to the attempts I tried?
推荐答案
我正在研究完全相同的问题，但还没有达到你的水平.
I'm working on exactly the same issue, but haven't got quite as far as you have.
您是否考虑过将像素直接绘制到画布上而不先将它们编码为 JPEG?OpenCV 工具包内部 http://sourceforge.net/projects/opencvlibrary/files/opencv-android/2.3.1/OpenCV-2.3.1-android-bin.tar.bz2/download (实际上并没有使用 opencv；别担心)，有一个名为 tutorial-0-androidcamera 的项目，它演示了将 YUV 像素转换为 RGB，然后将它们直接写入位图.
Have you considered drawing the pixels directly to the canvas without encoding them to JPEG first? Inside the OpenCV kit http://sourceforge.net/projects/opencvlibrary/files/opencv-android/2.3.1/OpenCV-2.3.1-android-bin.tar.bz2/download (which doesn't actually use opencv; don't worry), there's a project called tutorial-0-androidcamera that demonstrates converting the YUV pixels to RGB and then writing them directly to a bitmap.
相关代码本质上是:
public void onPreviewFrame(byte[] data, Camera camera, int width, int height) {
int frameSize = width*height;
int[] rgba = new int[frameSize+1];
// Convert YUV to RGB
for (int i = 0; i < height; i++)
for (int j = 0; j < width; j++) {
int y = (0xff & ((int) data[i * width + j]));
int u = (0xff & ((int) data[frameSize + (i >> 1) * width + (j & ~1) + 0]));
int v = (0xff & ((int) data[frameSize + (i >> 1) * width + (j & ~1) + 1]));
y = y < 16 ? 16 : y;
int r = Math.round(1.164f * (y - 16) + 1.596f * (v - 128));
int g = Math.round(1.164f * (y - 16) - 0.813f * (v - 128) - 0.391f * (u - 128));
int b = Math.round(1.164f * (y - 16) + 2.018f * (u - 128));
r = r < 0 ? 0 : (r > 255 ? 255 : r);
g = g < 0 ? 0 : (g > 255 ? 255 : g);
b = b < 0 ? 0 : (b > 255 ? 255 : b);
rgba[i * width + j] = 0xff000000 + (b << 16) + (g << 8) + r;
}
Bitmap bmp = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
bmp.setPixels(rgba, 0/* offset */, width /* stride */, 0, 0, width, height);
Canvas canvas = mHolder.lockCanvas();
if (canvas != null) {
canvas.drawBitmap(bmp, (canvas.getWidth() - width) / 2, (canvas.getHeight() - height) / 2, null);
mHolder.unlockCanvasAndPost(canvas);
} else {
Log.w(TAG, "Canvas is null!");
}
bmp.recycle();
}
当然，您必须对其进行调整以满足您的需求(例如，不为每一帧分配 rgba)，但这可能是一个开始.我很想看看它是否适合你——我目前仍在与与你正交的问题作斗争.
Of course you'd have to adapt it to meet your needs (ex. not allocating rgba each frame), but it might be a start. I'd love to see if it works for you or not -- i'm still fighting problems orthogonal to yours at the moment.
相关文章