redis数据同步工具redis-shake之两服务器之间数据实时同步案例
redis-shake是阿里开源的用于redis数据同步的工具,我这里主要用于两台redis服务器之间有些key数据要同步一下,所以在测试环境测试一下,写个教程以防下次使用的时候方便
安装redis这点就不用说了
介绍一下我的环境
centos7两台,redis一台4.0,一台5.0
安装redis-shake
下载列表:
https://github.com/alibaba/RedisShake
我这里下载的版本是release-v2.1.1-20210903
https://github.com/alibaba/RedisShake/releases/download/release-v2.1.1-20210903/release-v2.1.1-20210903.tar.gz
下载下来放入被同步的服务器上
我这里是root目录创建redis-shake文件夹 放进这里面,然后解压
[[email protected] redis-shake]# tar -zxvf release-v2.1.1-20210903.tar.gz
release-v2.1.1-20210903/
release-v2.1.1-20210903/redis-shake.linux
release-v2.1.1-20210903/redis-shake.conf
release-v2.1.1-20210903/redis-shake.windows
release-v2.1.1-20210903/redis-shake.darwin
[[email protected] redis-shake]# ll
总用量 58236
-rw-r--r-- 1 502 games 12589 8月 24 2022 redis-shake.conf
-rwxr-xr-x 1 502 games 13176492 9月 3 2021 redis-shake.darwin
-rwxr-xr-x 1 502 games 13235485 9月 3 2021 redis-shake.linux
-rwxr-xr-x 1 502 games 13356032 9月 3 2021 redis-shake.windows
-rw-r--r-- 1 root root 19842840 8月 24 12:49 release-v2.1.1-20210903.tar.gz
配置文件修改,记得把原来的redis-shake.conf备份一下加个.bak后缀
如下修改:
[[email protected] redis-shake]# cat redis-shake.conf
conf.version = 1
id = redis-shake
log.file = /root/redis-shake/redis-shake.log
log.level = info
pid_path =
system_profile = 9310
http_profile = 9320
parallel = 32
source.type = standalone
source.address = 172.18.1.24:63790
source.password_raw = 123456
source.auth_type = auth
source.tls_enable = false
source.rdb.input =
source.rdb.parallel = 0
source.rdb.special_cloud =
target.type = standalone
target.address = 127.0.0.1:6379
target.password_raw =
target.auth_type = auth
target.db = 0
key_exists = rewrite
filter.db.whitelist = 0
[[email protected] redis-shake]#
ps:
我的源redis服务器是172.18.1.24,端口:63790 密码:123456
同步至服务器192.168.1.98 ,端口:6379 密码:无
日志文件记录:/root/redis-shake/redis-shake.log
启动redis-shake服务
[[email protected] redis-shake]# ./redis-shake.linux -conf=redis-shake.conf -type=sync &
[2] 19824
ps:
这样就启动了,pid:19824,如果修改配置或者要重启服务可以使用kill命令 比如:
kill -9 19824
启动完了,看一下日志文件,我这就截一部分,看看都同步完了
[[email protected] redis-shake]# ll
总用量 58500
-rw-r--r-- 1 502 games 537 8月 24 13:58 redis-shake.conf
-rw-r--r-- 1 root root 12544 8月 24 13:52 redis-shake.conf.bak
-rwxr-xr-x 1 502 games 13176492 9月 3 2021 redis-shake.darwin
-rwxr-xr-x 1 502 games 13235485 9月 3 2021 redis-shake.linux
-rw-r--r-- 1 root root 136400 8月 24 14:15 redis-shake.log
-rw------- 1 root root 6 8月 24 13:55 redis-shake.pid
-rwxr-xr-x 1 502 games 13356032 9月 3 2021 redis-shake.windows
-rw-r--r-- 1 root root 19842840 8月 24 12:49 release-v2.1.1-20210903.tar.gz
[[email protected] redis-shake]# cat redis-shake.log
2022/08/24 13:55:40 [WARN] source.auth_type[auth] != auth
2022/08/24 13:55:40 [WARN] target.auth_type[auth] != auth
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] source rdb[172.18.1.24:63790] checksum[yes]
2022/08/24 13:55:40 [WARN]
______________________________
\ \ _ ______ |
\ \ / \___-=O'/|O'/__|
\ RedisShake, here we go !! \_______\ / | / )
/ / '/-==__ _/__|/__=-| -GM
/ Alibaba Cloud / * \ | |
/ / (o)
------------------------------
if you have any problem, please visit https://github.com/alibaba/RedisShake/wiki/FAQ
2022/08/24 13:55:40 [INFO] redis-shake configuration: {"ConfVersion":1,"Id":"redis-shake","LogFile":"/root/redis-shake/redis-shake.log","LogLevel":"info","SystemProfile":9310,"HttpProfile":9320,"Parallel":32,"SourceType":"standalone","SourceAddress":"172.18.1.24:63790","SourcePasswordRaw":"***","SourcePasswordEncoding":"***","SourceAuthType":"auth","SourceTLSEnable":false,"SourceRdbInput":[],"SourceRdbParallel":1,"SourceRdbSpecialCloud":"","TargetAddress":"127.0.0.1:6379","TargetPasswordRaw":"***","TargetPasswordEncoding":"***","TargetDBString":"0","TargetDBMapString":"","TargetAuthType":"auth","TargetType":"standalone","TargetTLSEnable":false,"TargetRdbOutput":"","TargetVersion":"5.0.9","FakeTime":"","KeyExists":"rewrite","FilterDBWhitelist":["0"],"FilterDBBlacklist":null,"FilterKeyWhitelist":null,"FilterKeyBlacklist":null,"FilterSlot":null,"FilterLua":false,"BigKeyThreshold":52428800,"Metric":false,"MetricPrintLog":false,"SenderSize":65535,"SenderCount":1024,"SenderDelayChannelSize":32,"KeepAlive":0,"PidPath":"","ScanKeyNumber":0,"ScanSpecialCloud":"","ScanKeyFile":"","Qps":500000,"ResumeFromBreakPoint":false,"Psync":true,"NCpu":0,"HeartbeatUrl":"","HeartbeatInterval":10,"HeartbeatExternal":"","HeartbeatNetworkInterface":"","ReplaceHashTag":false,"ExtraInfo":false,"SockFileName":"","SockFileSize":0,"FilterKey":null,"FilterDB":"","Rewrite":false,"SourceAddressList":["172.18.1.24:63790"],"TargetAddressList":["127.0.0.1:6379"],"SourceVersion":"4.0.6","HeartbeatIp":"127.0.0.1","ShiftTime":0,"TargetReplace":true,"TargetDB":0,"Version":"develop,cc226f841e2e244c48246ebfcfd5a50396b59710,go1.15.7,2021-09-03_10:06:55","Type":"sync","TargetDBMap":null}
2022/08/24 13:55:40 [INFO] DbSyncer[0] starts syncing data from 172.18.1.24:63790 to [127.0.0.1:6379] with http[9321], enableResumeFromBreakPoint[false], slot boundary[-1, -1]
2022/08/24 13:55:40 [INFO] DbSyncer[0] psync connect '172.18.1.24:63790' with auth type[auth] OK!
2022/08/24 13:55:40 [INFO] DbSyncer[0] psync send listening port[9320] OK!
2022/08/24 13:55:40 [INFO] DbSyncer[0] try to send 'psync' command: run-id[?], offset[-1]
2022/08/24 13:55:40 [INFO] Event:FullSyncStartId:redis-shake
2022/08/24 13:55:40 [INFO] DbSyncer[0] psync runid = a347eaedbfa83634774b5584ed0efe9d152c6d0f, offset = 0, fullsync
2022/08/24 13:55:40 [INFO] DbSyncer[0] rdb file size = 104942
2022/08/24 13:55:40 [INFO] Aux information key:redis-ver value:4.0.6
2022/08/24 13:55:40 [INFO] Aux information key:redis-bits value:64
2022/08/24 13:55:40 [INFO] Aux information key:ctime value:1661320637
2022/08/24 13:55:40 [INFO] Aux information key:used-mem value:2231719
2022/08/24 13:55:40 [INFO] Aux information key:repl-stream-db value:0
2022/08/24 13:55:40 [INFO] Aux information key:repl-id value:a347eaedbfa83634774b5584ed0efe9d152c6d0f
2022/08/24 13:55:40 [INFO] Aux information key:repl-offset value:0
2022/08/24 13:55:40 [INFO] Aux information key:aof-preamble value:0
2022/08/24 13:55:40 [INFO] db_size:20 expire_size:14
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] db_size:2 expire_size:0
2022/08/24 13:55:40 [INFO] Aux information key:lua value:return redis.call('exists',KEYS[1])<1 and redis.call('setex',KEYS[1],ARGV[2],ARGV[1])
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] input password is empty, skip auth address[127.0.0.1:6379] with type[auth].
2022/08/24 13:55:40 [INFO] warning, rewrite key: name
2022/08/24 13:55:40 [INFO] DbSyncer[0] total = 102.48KB - 102.48KB [100%] entry=20 filter=3
2022/08/24 13:55:40 [INFO] DbSyncer[0] sync rdb done
...
上面就同步过来了,如果日志没报错的话就ok了;
用redis客户端软件打开检查测试一下
然后在源redis上修改一下数据,在同步redis服务器上看看,同步过来就ok了
测试我就不截图了,比较简单,有兴趣有需要的可以自行测试一下
相关文章