EsgynDB Troubleshooting-*** ERROR[8837] Internal e

现象

trafci或JDBC连接数据库突然出现卡顿的现象，卡顿之后报错*** ERROR[8837] Internal error occurred.，

[trafodion@uatzf04 scripts]$ trafci

Welcome to QianBase Command Interface 
Copyright (c) 2015-2019 Esgyn Corporation

Host Name/IP Address: uatzf01:23400,uatzf02:23400,uatzf03:23400
User Name: trafodion

*** ERROR[8837] Internal error occurred. User: TRAFODION  [2019-07-29 16:31:35]

分析

*** ERROR[8837] Internal error occurred.判断可能是数据库与LDAP连接出现了问题，使用ldapcheck --username=trafodion检查是否成功，发现同样出现卡顿的现象。因此判断就是与LDAP连接出现了问题。

[trafodion@uatzf04 scripts]$ ldapcheck --username=trafodion
Lookup request: externalName trafodion, configName 'not available' (configNumber -2), result 2 (Unexpected error returned from LDAP)

检查slapd服务状态，

[root@uatzf01 ~]# service slapd status
Redirecting to /bin/systemctl status slapd.service
● slapd.service - OpenLDAP Server Daemon
   Loaded: loaded (/usr/lib/systemd/system/slapd.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-07-24 09:17:12 CST; 5 days ago
     Docs: man:slapd
           man:slapd-config
           man:slapd-hdb
           man:slapd-mdb
           file:///usr/share/doc/openldap-servers/guide.html
 Main PID: 42947 (slapd)
   CGroup: /system.slice/slapd.service
           └─42947 /usr/sbin/slapd -u ldap -h ldapi:/// ldap:///

Jul 29 16:16:37 uatzf01.esgyn.cn slapd[42947]: warning: cannot open /etc/hosts.allow: Too many open files
Jul 29 16:16:37 uatzf01.esgyn.cn slapd[42947]: warning: cannot open /etc/hosts.deny: Too many open files
Jul 29 16:16:37 uatzf01.esgyn.cn slapd[42947]: fd=1023 DENIED from unknown (172.31.234.13)
Jul 29 16:16:41 uatzf01.esgyn.cn slapd[42947]: warning: cannot open /etc/hosts.allow: Too many open files
Jul 29 16:16:41 uatzf01.esgyn.cn slapd[42947]: warning: cannot open /etc/hosts.deny: Too many open files
Jul 29 16:16:41 uatzf01.esgyn.cn slapd[42947]: fd=1023 DENIED from unknown (172.31.234.12)
Jul 29 16:16:42 uatzf01.esgyn.cn slapd[42947]: warning: cannot open /etc/hosts.allow: Too many open files
Jul 29 16:16:42 uatzf01.esgyn.cn slapd[42947]: daemon: accept(8) failed errno=24 (Too many open files)
Jul 29 16:16:42 uatzf01.esgyn.cn slapd[42947]: warning: cannot open /etc/hosts.deny: Too many open files
Jul 29 16:16:42 uatzf01.esgyn.cn slapd[42947]: fd=1023 DENIED from unknown (172.31.234.11)

我们可以看到“Too many open files”之类的错误，说明ldap用户的max open file已经达到上限。

解决

解决方案如博客Linux OpenLAP 修改max open files