使用 OpenCV 进行视频稳定

2021-12-10 00:00:00 opencv image-processing c++ video

我有一个视频源，它是用移动的相机拍摄的，包含移动的物体.我想稳定视频，以便所有静止物体在视频源中保持静止.我如何使用 OpenCV 做到这一点?

I have a video feed which is taken with a moving camera and contains moving objects. I would like to stabilize the video, so that all stationary objects will remain stationary in the video feed. How can I do this with OpenCV?

即例如，如果我有两个图像 prev_frame 和 next_frame，我如何转换 next_frame 以使摄像机看起来静止?

i.e. For example, if I have two images prev_frame and next_frame, how do I transform next_frame so the video camera appears stationary?

推荐答案

我可以建议以下解决方案之一:

I can suggest one of the following solutions:

  1. 使用局部高级特征:OpenCV 包含 SURF，因此:对于每一帧，提取 SURF 特征.然后构建特征 Kd-Tree(也在 OpenCV 中)，然后匹配每两个连续帧以找到对应的特征对.将这些对输入 cvFindHomography 以计算这些帧之间的单应性.根据(组合..)单应性扭曲帧以稳定.据我所知，这是一种非常强大和复杂的方法，但是 SURF 提取和匹配可能会很慢
  2. 如果您预计两帧之间只有很小的移动，例如使用 Harris 角点检测并在两帧中构建彼此最接近的角点对，然后如上提供给 cvFindHomography.可能更快，但不够稳健.
  3. 如果您将移动限制为翻译，您可能可以用更...简单的东西替换 cvFindHomography，以获取特征对之间的翻译(例如平均值)
  4. 使用相位相关(参考http://en.wikipedia.org/wiki/Phase_correlation)，如果您只希望在两个帧之间进行转换.OpenCV 包括 DFT/FFT 和 IFFT，请参阅链接的维基百科关于公式和解释的文章.
  1. Using local high level features: OpenCV includes SURF, so: for each frame, extract SURF features. Then build feature Kd-Tree (also in OpenCV), then match each two consecutive frames to find pairs of corresponding features. Feed those pairs into cvFindHomography to compute the homography between those frames. Warp frames according to (combined..) homographies to stabilize. This is, to my knowledge, a very robust and sophisticated approach, however SURF extraction and matching can be quite slow
  2. You can try to do the above with "less robust" features, if you expect only minor movement between two frames, e.g. use Harris corner detection and build pairs of corners closest to each other in both frames, feed to cvFindHomography then as above. Probably faster but less robust.
  3. If you restrict movement to translation, you might be able to replace cvFindHomography with something more...simple, to just get the translation between feature-pairs (e.g. average)
  4. Use phase-correlation (ref. http://en.wikipedia.org/wiki/Phase_correlation), if you expect only translation between two frames. OpenCV includes DFT/FFT and IFFT, see the linked wikipedia article on formulas and explanation.

编辑我最好明确指出三点，以防万一:

EDIT Three remarks I should better mention explicitly, just in case:

  1. 基于单应性的方法可能非常精确，因此静止物体将保持静止.但是，单应性还包括透视失真和缩放，因此结果可能看起来有点……不常见(甚至因某些快速移动而失真).虽然准确，但这可能不太美观；因此，请将此用于进一步处理或取证等.但是您应该尝试一下，对于某些场景/动作也可能非常满意.
  2. 据我所知，至少有几个免费的视频稳定工具使用相位相关.如果您只想不摇晃"相机，这可能更可取.
  3. 该领域正在进行大量研究.您会在一些论文中发现一些更复杂的方法(尽管它们可能需要的不仅仅是 OpenCV).

相关文章