terminatingtheinstanceduetoerror119-创新互联

RAC安装过程一切正常，关机重启后，GI启动正常，实例没起来。

[root@rac1 network-scripts]# crsctl stat res -t

--------------------------------------------------------------------------------

NAME      TARGET  STATE     SERVER          STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.DATA.dg

        ONLINE  ONLINE    rac1

        ONLINE  ONLINE    rac2

ora.FRA.dg

        ONLINE  ONLINE    rac1

        ONLINE  ONLINE    rac2

ora.LISTENER.lsnr

        ONLINE  ONLINE    rac1

        ONLINE  ONLINE    rac2

ora.OCR.dg

        ONLINE  ONLINE    rac1

        ONLINE  ONLINE    rac2

ora.asm

        ONLINE  ONLINE    rac1           Started

        ONLINE  ONLINE    rac2           Started

ora.gsd

        OFFLINE OFFLINE    rac1

        OFFLINE OFFLINE    rac2

ora.net1.network

        ONLINE  ONLINE    rac1

        ONLINE  ONLINE    rac2

ora.ons

        ONLINE  ONLINE    rac1

        ONLINE  ONLINE    rac2

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

   1     ONLINE  ONLINE    rac1

ora.LISTENER_SCAN2.lsnr

   1     ONLINE  ONLINE    rac2

ora.LISTENER_SCAN3.lsnr

   1     ONLINE  ONLINE    rac2

ora.cvu

   1     ONLINE  ONLINE    rac2

ora.oc4j

   1     ONLINE  ONLINE    rac2

ora.rac.db

   1     ONLINE  OFFLINE                Instance Shutdown

   2     ONLINE  OFFLINE                Instance Shutdown

ora.rac1.vip

   1     ONLINE  ONLINE    rac1

ora.rac2.vip

   1     ONLINE  ONLINE    rac2

ora.scan1.vip

   1     ONLINE  ONLINE    rac1

ora.scan2.vip

   1     ONLINE  ONLINE    rac2

ora.scan3.vip

   1     ONLINE  ONLINE    rac2

马上查数据库日志，有如下发现

[oracle@rac1 trace]$ tail -200f alert_RAC1.log

USER (ospid: 3610): terminating the instance due to error 119

Instance terminated by USER, pid = 3610

强行启动行不行呢？

[root@rac1 ~]# srvctl start database -d RAC

PRCR-1079 : 无法启动资源 ora.rac.db

CRS-5017: The resource action "ora.rac.db start" encountered the following error:

ORA-00119: invalid specification for system parameter REMOTE_LISTENER

ORA-00132: syntax error or unresolved network name 'rac-scan.us.oracle.com:1521'

. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/rac2/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-5017: The resource action "ora.rac.db start" encountered the following error:

ORA-00119: invalid specification for system parameter REMOTE_LISTENER

ORA-00132: syntax error or unresolved network name 'rac-scan.us.oracle.com:1521'

. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/rac1/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2674: Start of 'ora.rac.db' on 'rac2' failed

CRS-2674: Start of 'ora.rac.db' on 'rac1' failed

CRS-2632: There are no more servers to try to place resource 'ora.rac.db' on that would satisfy its placement policy

注意到报错信息中提到ORA-00132: syntax error or unresolved network name 'rac-scan.us.oracle.com:1521'，是不是呢？

反正和网络有关先tnsping一下

[grid@rac1 ~]$ tnsping rac2

TNS Ping Utility for Linux: Version 11.2.0.4.0 - Production on 09-DEC-2015 00:56:12

Copyright (c) 1997, 2013, Oracle.  All rights reserved.

Used parameter files:

/u01/app/11.2.0/grid/network/admin/sqlnet.ora

Used EZCONNECT adapter to resolve the alias

Attempting to contact (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.56.12)(PORT=1521)))

OK (0 msec)

看来到监听是没问题的

不放心，再看一眼tnsnames.ora

[root@rac1 ~]# cat /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/tnsnames.ora

# tnsnames.ora Network Configuration File: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/tnsnames.ora

# Generated by Oracle configuration tools.

RAC =

 (DESCRIPTION =

  (ADDRESS = (PROTOCOL = TCP)(HOST = rac-scan.us.oracle.com)(PORT = 1521))

  (CONNECT_DATA =

   (SERVER = DEDICATED)

   (SERVICE_NAME = RAC)

  )

 )

[root@rac1 ~]# ping rac-scan.us.oracle.com

ping: unknown host rac-scan.us.oracle.com

这就有问题了，我在DNS中配置的域名怎么会不可达呢？安装时一切功能都正常的呀！

[root@rac1 ~]# nslookup rac-scan.us.oracle.com

Server: 192.168.1.1

Address: 192.168.1.1#53

** server can't find rac-scan.us.oracle.com: NXDOMAIN

更有问题了，DNS服务器怎么成192.168.1.1了呢？

依次对照所有的DNS配置文件，发现了一个异常的

[root@rac1 ~]# cat /etc/resolv.conf

; generated by /sbin/dhclient-script

nameserver 192.168.1.1

nameserver 192.168.0.1

怎么回到了操作系统最初始的配置了呢？这就要提到另外一个配置文件

[root@rac2 network-scripts]# cat ifcfg-eth0

DEVICE=eth0

TYPE=Ethernet

ONBOOT=yes

NM_CONTROLLED=yes

BOOTPROTO=dhcp

DEFROUTE=yes

PEERDNS=yes

PEERROUTES=yes

IPV4_FAILURE_FATAL=yes

IPV6INIT=no

NAME="System eth0"

USERCTL=no

注意到其中的PEERDNS=yes这一项了吗？

在5.0时代DNS服务器写在 /etc/resolv.conf 文件中，但到了6.0时代DNS可以写在/etc/resolv.conf但是此时需要在 /etc/sysconfig/network-scripts/ifcfg-eth0 文件中添加 PEERDNS=no 配置，不然每次重启网卡就会重写/etc/resolv.conf文件的内容。

http://blog.chinaunix.net/uid-26495963-id-3230810.html

另外有需要云服务器可以了解下创新互联cdcxhl.cn，海内外云服务器15元起步，三天无理由+7*72小时售后在线，公司持有idc许可证，提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案，具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势，专为企业上云打造定制，能够满足用户丰富、多元化的应用场景需求。