如何使用 JNI 在 C 中获取原始 Android 相机缓冲区?

2022-01-08 00:00:00 c camera android java java-native-interface

我一直在彻底搜索 Google 和 StackOverflow，但找不到这个.也许我错过了一些明显的东西.谢谢！

I've been searching Google and StackOverflow exhaustively and cannot find this. Maybe I'm missing something obvious. Thanks!

(这是因为预览回调的Java实现[即使有缓冲区]效率太低.)

(This is because the Java implementation of the preview callback [even with buffer] is too inefficient.)

推荐答案

我对主题做了一点调查.这个介绍(来自p.277，中文)帮助很大.

I made a little investigation on topic. This presentation (from p.277, Chinese) helped a lot.

正如其他人提到的，您可以使用 Camera.setPreviewCallback 方法获取缓冲区.
以下是它的发生方式(详细版本):

As others mentioned, you can get a buffer using a Camera.setPreviewCallback method.
Here's how it happens there (a verbose version):

  1. 用户调用 Camera.startPreview() 这是一个原生函数.
  2. android_hardware_Camera_startPreview 调用C++ Camera 类的startPreview 方法.
  3. Camera调用ICamera接口的startPreview方法
  4. ICamera 对远程客户端进行 IPC 调用.
  5. 调用CameraService类的setCameraMode方法.
  6. CameraService 设置一个窗口显示预览并调用CameraHardwareInterface 类的startPreview 方法.
  7. 后者尝试在特定 camera_device_t 设备上调用 start_preview 方法.
    我没有进一步查找，但它应该会调用驱动程序.
  8. 当图片到达时，调用CameraService的dataCallback.
  9. 将数据传递给客户端的handlePreviewData方法.
  10. 客户端要么复制缓冲区，要么直接将其发送到 ICameraClient.
  11. ICameraClient 通过IPC 将其发送到Camera.
  12. Camera 调用已注册的监听器并将缓冲区传递给 JNI.
  13. 它调用 Java 类中的回调.如果用户使用 Camera.addCallbackBuffer 提供了一个缓冲区，则它首先复制到缓冲区.
  14. 最后，Java 类 Camera 处理消息并调用 Camera.PreviewCallback 的 onPreviewFrame 方法.
  1. User calls Camera.startPreview() which is a native function.
  2. android_hardware_Camera_startPreview calls startPreview method of C++ Camera class.
  3. Camera calls a startPreview method of ICamera interface
  4. ICamera makes an IPC call to remote client.
  5. It calls a setCameraMode method of CameraService class.
  6. CameraService sets a window to display a preview and calls a startPreview method of CameraHardwareInterface class.
  7. The latter tries to call a start_preview method on particular camera_device_t device.
    I didn't looked up further but it should perform a call to the driver.
  8. When image arrives, dataCallback of CameraService is invoked.
  9. It passes data to handlePreviewData method of client.
  10. Client either copies the buffer or sends it directly to the ICameraClient.
  11. ICameraClient sends it over IPC to the Camera.
  12. Camera calls a registered listener and passes buffer to JNI.
  13. It invokes a callback in Java class. If user provided a buffer with Camera.addCallbackBuffer then it copies to the buffer first.
  14. Finally Java class Camera handles the message and invokes a onPreviewFrame method of Camera.PreviewCallback.

如您所见，调用了 2 个 IPC 调用，并且在步骤 10、11 中至少复制了两次缓冲区.camera_device_t 返回的原始缓冲区的第一个实例是托管在另一个进程中，由于 CameraService 中的安全检查，您无法访问它.

As you can see 2 IPC calls were invoked and buffer was copied at least twice on steps 10, 11. First instance of raw buffer which is returned by camera_device_t is hosted in another process and you cannot access it due to security checks in CameraService.

但是，当您使用 Camera.setPreviewTexture 或 Camera.setPreviewDisplay 设置预览表面时，它会直接传递给相机设备并实时刷新，而无需上面的所有链条.正如它的文档所说:

However, when you set a preview surface using either Camera.setPreviewTexture or Camera.setPreviewDisplay it is be passed directly to the camera device and refreshed in realtime without participation of all the chain above. As it's documentation says:

处理一个由屏幕合成器管理的原始缓冲区.

Handle onto a raw buffer that is being managed by the screen compositor.

Java 类 Surface 有一个方法来检索它的内容:

Java class Surface has a method to retrieve it's contents:

public static native Bitmap screenshot(int width, int height, int minLayer, int maxLayer);

但是这个 API 是隐藏的.参见 这个问题 了解使用方法.

But this API is hidden. See i.e. this question for a way to use it.

相关文章