同一节点启动多个bigchaindb实例
需求
某个节点处于两个不同的bigchaindb集群，或者说一个节点需要启动两个不同的bigchaindb实例
问题
当前bigchaindb版本
BigchainDB (1.0.0rc1)
bigchaindb-driver (0.3.1)
1
2
原本的bigchiandb配置文件（/root/.bigchaindb）
{
"database": {
"connection_timeout": 5000,
"max_tries": 3,
"replicaset": "bigchain-rs",
"name": "bigchain",
"host": "localhost",
"login": null,
"password": null,
"ssl": false,
"port": 27017,
"backend": "mongodb"
},
"log": {
"level_logfile": "info",
"datefmt_logfile": "%Y-%m-%d %H:%M:%S",
"granular_levels": {},
"file": "/root/bigchaindb.log",
"fmt_logfile": "[%(asctime)s] [%(levelname)s] (%(name)s) %(message)s (%(processName)-10s - pid: %(process)d)",
"error_file": "/root/bigchaindb-errors.log",
"fmt_console": "[%(asctime)s] [%(levelname)s] (%(name)s) %(message)s (%(processName)-10s - pid: %(process)d)",
"level_console": "info",
"datefmt_console": "%Y-%m-%d %H:%M:%S"
},
"keypair": {
"private": "BAjL63tqdknLbhXB9vP9F9sgaA699z1CgsMKSKjzFsAg",
"public": "PKEsj5XCPZjbbsRraixPriPdJ3j9wm1GrMxkcNUAR5n"
},
"wsserver": {
"port": 9985,
"host": "localhost"
},
"server": {
"threads": null,
"workers": null,
"loglevel": "info",
"bind": "localhost:9984"
},
"keyring": [],
"backlog_reassign_delay": 120
}
利用该配置文件启动一个实例
bigchaindb -c /root/.bigchaindb start
1
复制一份配置文件，修改其中需要的端口（9984->9982,9985->9983），
cp .bigchaindb .bigchaindb2
sed -i "s/9985/9983/g" .bigchaindb2
sed -i "s/9984/9982/g" .bigchaindb2
再启动一个实例时报错：
root@clean:~# bigchaindb -c .bigchaindb2 start
INFO:bigchaindb.config_utils:Configuration loaded from `.bigchaindb2`
Traceback (most recent call last):
File "/usr/local/bin/bigchaindb", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.4/dist-packages/bigchaindb/commands/bigchaindb.py", line 348, in main
utils.start(create_parser(), sys.argv[1:], globals())
File "/usr/local/lib/python3.4/dist-packages/bigchaindb/commands/utils.py", line 204, in start
return func(args)
File "/usr/local/lib/python3.4/dist-packages/bigchaindb/commands/utils.py", line 49,in configure
command(args)
File "/usr/local/lib/python3.4/dist-packages/bigchaindb/commands/utils.py", line 75,in start_logging
setup_logging(user_log_config=config.get('log'))
File "/usr/local/lib/python3.4/dist-packages/bigchaindb/log/setup.py", line 49, in setup_logging
setup_sub_logger(user_log_config=user_log_config)
File "/usr/local/lib/python3.4/dist-packages/bigchaindb/log/setup.py", line 38, in setup_sub_logger
server = LogRecordSocketServer()
File "/usr/local/lib/python3.4/dist-packages/bigchaindb/log/setup.py", line 156, in __init__
super().__init__((host, port), handler)
File "/usr/lib/python3.4/socketserver.py", line 430, in __init__
self.server_bind()
File "/usr/lib/python3.4/socketserver.py", line 444, in server_bind
self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
root@clean:~#
方案
定位到错误位置：/usr/local/lib/python3.4/dist-packages/bigchaindb/log/setup.py
log的setup函数为setup_logging，其中启动了publisher与subscriber。
def setup_logging(*, user_log_config=None):
setup_pub_logger()
setup_sub_logger(user_log_config=user_log_config)
log的publisher调用logging.handlers.SocketHandler将远程输出日志到TCP/IP sockets。注意其中使用了写死的端口DEFAULT_SOCKET_LOGGING_PORT
DEFAULT_SOCKET_LOGGING_HOST = 'localhost'
DEFAULT_SOCKET_LOGGING_PORT = DEFAULT_TCP_LOGGING_PORT
def setup_pub_logger():
dictConfig(PUBLISHER_LOGGING_CONFIG)
socket_handler = logging.handlers.SocketHandler(
DEFAULT_SOCKET_LOGGING_HOST, DEFAULT_SOCKET_LOGGING_PORT)
socket_handler.setLevel(logging.DEBUG)
logger = logging.getLogger()
logger.addHandler(socket_handler)
DEFAULT_TCP_LOGGING_PORT在logging模块handlers.py中定义
DEFAULT_TCP_LOGGING_PORT = 9020
1
同时，对于日志的subscriber来说，bigchindb启动一个进程来启动subscriber，进程的target为LogRecordSocketServer.serve_forever
def setup_sub_logger(*, user_log_config=None):
server = LogRecordSocketServer()
with server:
server_proc = Process(
target=server.serve_forever,
kwargs={'log_config': user_log_config},
)
server_proc.start()
该类的__init__里实际上已经指定了订阅的端口，即logging.handlers.DEFAULT_TCP_LOGGING_PORT
class LogRecordSocketServer(ThreadingTCPServer):
"""
Simple TCP socket-based logging server.
"""
allow_reuse_address = True
def __init__(self,
host='localhost',
port=logging.handlers.DEFAULT_TCP_LOGGING_PORT,
handler=LogRecordStreamHandler):
super().__init__((host, port), handler)
def serve_forever(self, *, poll_interval=0.5, log_config=None):
sub_logging_config = create_subscriber_logging_config(
user_log_config=log_config)
dictConfig(sub_logging_config)
try:
super().serve_forever(poll_interval=poll_interval)
except KeyboardInterrupt:
pass
至此，我们可以看到，虽然我们bigchaindb的配置文件进行了修改，使得两个实例的bigchaindb端口没有被占用，但实际上两个实例的log的publisher都采用同一个端口
commands/utils.py调用了setup_logging函数
def start_logging_process(command):
@functools.wraps(command)
def start_logging(args):
from bigchaindb import config
setup_logging(user_log_config=config.get('log'))
command(args)
return start_logging
start_logging_process由run_start通过注解的方式进行调用，并同时调用了configure_bigchaindb，继续跟踪，bigchaindb依次调用了bigchaindb.config_utils.autoconfigure->env_config
@configure_bigchaindb
@start_logging_process
def run_start(args):
根据env_config的注释，该函数将config配置文件加载为map，判断是否存在对应的环境变量，否则赋值为配置文件中的值，并后将配置信息存储在bigchaindb.config中。至此，我们则可以通过在配置文件中加入一个用来配置publisher与subscriber的端口，再在代码中调用bigchaindb.config即可
解决
修改.bigchaindb配置文件，在key值为log下再增加一项
{
"database": {
...
},
"log": {
"port": 9986,
"level_logfile": "info",
...
修改
def setup_logging(*, user_log_config=None):
logging.info(">>> publisher port is")
try:
import bigchaindb
global DEFAULT_SOCKET_LOGGING_PORT
DEFAULT_SOCKET_LOGGING_PORT = bigchaindb.config['log']['port']
except:
pass
logging.info(DEFAULT_SOCKET_LOGGING_PORT)
setup_pub_logger()
setup_sub_logger(user_log_config=user_log_config)
def setup_sub_logger(*, user_log_config=None):
server = LogRecordSocketServer(port=DEFAULT_SOCKET_LOGGING_PORT)
with server:
server_proc = Process(
target=server.serve_forever,
kwargs={'log_config': user_log_config},
)
server_proc.start()
再复制一份配置文件：
cp .bigchaindb .bigchaindb2
sed -i "s/9984/9981/g" .bigchaindb2
sed -i "s/9985/9982/g" .bigchaindb2
sed -i "s/9986/9983/g" .bigchaindb2
# 修改数据库名
sed -i "s/\(\"name\": \"\).*/\1bigchain2\"/g" .bigchaindb2
可以启动两个bigchaindb实例了
bigchaindb -c .bigchaindb start
bigchaindb -c .bigchaindb2 start
使用python连接时分别用9984与9981端口即可
————————————————
版权声明：本文为CSDN博主「lwyeluo」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/lwyeluo/article/details/74003642
相关文章