安装完集群四套库都报ORA-15025 27041跟踪文件报ORA-17503 27300-303

    近用户安装集群，数据库版本19C ，采用ASM存储，udev绑定的方式实现，但是部署后当天晚上2点开始报错错误如下所示
WARNING: group 2 file 262 block 4876 read failed, OSD error 1129075524.
2023-03-15T02:04:20.423765+08:00
Errors in file /oracle/diag/rdbms/rac03nf/rac03nf1/trace/rac03nf1_ora_21392.trc:
ORA-15025: could not open disk "/dev/mapper/rac03nf_vmax400k-1_data_500G-3"
ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied
Additional information: 3

再打开trace文件可以看到如下内容重复出现
ORA-17503: ksfdopn:2 Failed to open file +DATA1/RAC03NF/PASSWORD/pwdrac03nf.256.1130920523
ORA-27300: OS system dependent operation:open failed with status: 13 《〈〈〈〈open失败，状态号13
ORA-27301: OS failure message: Permission denied
ORA-27302: failure occurred at: sskgmsmr_7
ORA-17503: ksfdopn:2 Failed to open file +DATA1/RAC03NF/PASSWORD/pwdrac03nf.256.1130920523
ORA-27300: OS system dependent operation:open failed with status: 13
ORA-27301: OS failure message: Permission denied
ORA-27302: failure occurred at: sskgmsmr_7
<error barrier> at 0x7fffffff4128 placed kzia.c@2649
ORA-01017: invalid username/password; logon denied
*** CLIENT DRIVER:(SQL*PLUS) 2023-03-15T02:04:20.412681+08:00
如下内容在读到不同ASM磁盘时都会报错，这里是warning。
WARNING: failed to open a disk[/dev/mapper/rac03nf_vmax400k-1_data_500G-3]
ORA-15025: could not open disk "/dev/mapper/rac03nf_vmax400k-1_data_500G-3"
ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied
Additional information: 3
<error barrier> at 0x7ffffffecbe0 placed kfksubs.c@5258
kfk_debug_get_user_groups: uid:900, euid:900, gid:900, egid:908

我们找了一些资料，但是都没有找到完全以一样的报错，也给我们一些提示比如文档Doc ID2310640.1 这个文档说明
如果连接数据库的的用户不是oinstall组，则会报ORA-17503 27300 27301 27302 01017错误，所以我们查询了用户属组信息
[grid@xxx-222 ~]$id oracle
uid=900(oracle) gid=900(oinstall) groups=900(oinstall),901(dba),902(oper),903(backupdba),904(dgdba),905(kmdba),906(asmdba),909(racdba)
[grid@xxx-222 ~]$id grid
uid=901(grid) gid=900(oinstall) groups=900(oinstall),906(asmdba),907(asmoper),908(asmadmin),909(racdba)

这里显示应该没有问题
我们把问题回到错误ORA-27300: OS system dependent operation:open failed with status: 13 ，这个status 13 查询MOS的解释是
STATUS 13 - EACCES Permission denied .
An attempt was made to access a forbidden file by the protection system.
总之按照Oracle观点不允许访问。具体哪里出问题，我继续通过错误提示直接在MOS搜索，文档Doc ID 2347696.1似乎给我们更多提示
他认为由于$ORACLE_HOME/bin/oracle的标志位没有设置导致
比如问题文件如下
[oracle@xxx-222 ~]$ls -lrt $ORACLE_HOME/bin/oracle
-rwxr-x--x 1 oracle asmadmin 450028544 Mar 8 09:45 /oracle/19/bin/oracle
而正确的二进制文件属性如下所示
[oracle@xxx-222 ~]$ls -lrt $ORACLE_HOME/bin/oracle
-rwsr-s--x 1 oracle asmadmin 450028544 Mar 8 09:45 /oracle/19/bin/oracle
修改方式
1 #crsctl stop crs
2 $chmod 6751 $ORACLE_HOME/bin/oracle
3 #crsctl start crs
我们查询了当前库的情况的这个二进制文件属性
oracle用户
[oracle@xxx-222 ~]$ls -lrt $ORACLE_HOME/bin/oracle
-rwsr-s--x 1 oracle asmadmin 450028544 Mar 8 09:45 /oracle/19/bin/oracle
grid用户
[grid@xxx-222 ~]$ls -lrt $ORACLE_HOME/bin/oracle
-rwsr-s--x 1 grid oinstall 427366696 Mar 8 09:49 /grid/19/bin/oracle

发现二者的属组是不一致的，因为Oracle用户需要有权限打开或者读取ASM磁盘文件，下面我们看看UDV绑定的ASM磁盘信息
ASM配置规则文件的属组
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-360000970000297200336533030303731",OWNER="grid",SYMLINK+="rac03nf_vmax400k-2_data_500G-30",GROUP="oinstall",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-360000970000297200335533030303842",OWNER="grid",SYMLINK+="rac03nf_vmax400k-3_asm_100G-1",GROUP="oinstall",MODE="0660"

显然这里是grid.oinstall 而我们的Oralce用户下的属组如下
[oracle@xxx-222 ~]$ls -lrt $ORACLE_HOME/bin/oracle
-rwsr-s--x 1 oracle asmadmin 450028544 Mar 8 09:45 /oracle/19/bin/oracle
所以这里需要改成一直

chown oracle.oinstall /oracle/19/bin/oracle