在 Debezium Mysql Connector 中将更多表列入白名单的有效方法

2021-11-12 00:00:00 mysql-connector mysql apache-kafka-connect debezium

是否有任何最佳实践可用于将新表列入 debezium mysql 连接器的白名单?

Are there any best practices that are followed for whitelisting a new table to the debezium mysql connector ?

我们正在为 CDC 流使用 debezium mysql 连接器，并且出现了一个用例，将更多表列入连接器配置的白名单.这是正在使用的 debezium 的版本详细信息和

We are using debezium mysql connector for our CDC flows and a use case has arised to whitelist more tables to the connector configuration. Here are the version details of the debezium being used and the 

{
    "class": "io.debezium.connector.mysql.MySqlConnector",
    "version": "0.8.0.Final",
    "snapshot.mode": "schema_only"
}

有一张 debezium 票 https://issues.redhat.com/browse/DBZ-906 表示在

There is a debezium ticket https://issues.redhat.com/browse/DBZ-906 which indicates to run the connector in 

"snapshot.mode": "schema_only_recover"

mode 并在重新启动连接器之前重新创建历史主题.我有几个关于这种方法的问题

mode and to recreate the history topic before restarting the connector. I had a couple of questions regarding this approach

  • 这个选项是否也适用于上述版本?
  • 如何获取在表被列入白名单之前对新表进行的更新?

  • Is this option available for the above mentioned version as well ?
  • How can I get the updates to the new table made before the table was whitelisted ?

    推荐答案

    首先，请升级 Debezium，因为 0.8.0 版本太旧了.

    first of all, pleas eupgrade Debezium as version 0.8.0 is very old.

    通常有多种方法可以解决您的问题，但如果您能负担得起流式传输的暂停时间，那么我建议您这样做1) 停止当前连接器2) 使用列入白名单的新表启动新连接器以在其上执行快照3) 快照完成后，使用更新后的白名单列表启动原始连接器

    Generally there are multiple approaches to solving your issue but if you can afford a pause in streaming than I'd recommend to 1) Stop the current connector 2) Start a new connector with whitelisted new table to execute the snapshot on it 3) When snapshot is completed then start the original connector with updated whitelist table list

    参见例如 https://groups.google.com/forum/?fromgroups=#!msg/debezium/Iw4BgLZ8Mq4/SZBLGwEaAQAJ;context-place=forum/debezium 了解详细信息

    See for example https://groups.google.com/forum/?fromgroups=#!msg/debezium/Iw4BgLZ8Mq4/SZBLGwEaAQAJ;context-place=forum/debezium for exact details

    如果您无法承受停机时间，请查看 DBZ-175 中介绍的实验方法一个>

    If you cannot afford the downtime then please check the experimental approach introduced in DBZ-175

    • 相关文章