重置 JDBC Kafka 连接器以从一开始就开始拉行?
Kafka 连接器可以使用主键和时间戳来确定需要处理哪些行.
The Kafka Connector can make use of a primary key and a timestamp to determine which rows need to be processed.
我正在寻找一种重置连接器的方法，以便它从一开始就进行处理.
I'm looking for a way to reset the Connector so that it will process from the beginning of time.
推荐答案
因为要求是在分布式模式下运行，所以最简单的做法是将连接器名称更新为新值.这将提示在 connect-offsets 主题中创建一个新条目，因为它看起来像一个全新的连接器.然后连接器应该再次开始读取，好像还没有向 Kafka 写入任何内容.您还可以手动向与该特定连接器关联的连接偏移主题中的键发送墓碑消息，但重命名比处理它容易得多.此方法适用于所有源连接器，而不仅仅是此处描述的 JDBC 连接器.
Because the requirement is to run in distributed mode, the easiest thing to do is to update the connector name to a new value. This will prompt a new entry to be made into the connect-offsets topic as it looks like a totally new connector. Then the connector should start reading again as if nothing has been written to Kafka yet. You could also manually send a tombstone message to the key in the connect-offsets topic associated with that particular connector, but renaming is much easier than dealing with that. This method applies to all source connectors, not only the JDBC one described here.
相关文章