这篇文章其实在草稿箱里面躺了快1年了,只是太长,长的我都没有信心完成它了。不过,放着可惜,还是拆分一下share出来吧。
关于flex cluster
1. 有hub node和leaf node的概念,目前数据库只能放在hub node,leaf node据说是用于放耦合度较低的服务,如weblogic。 2. flex和非flex可以互相转换crsctl set cluster flex|standard, 需重启crs 3. hub和leaf可以互相转换 4. 必须GNS固定VIP
之前有消息说db可以安装在leaf node,甚至有人写书都写出来说可以安装在leaf node上,因为有flex asm的关系,可以接收来自非本主机的db实例。但是事实证明,等正式发布的这个版本,db是不能安装在leaf node,如果尝试安装,会报错:
1.flex cluster安装。
flex cluster的安装,其实只要做好了准备工作,安装难度并不大。只是12c中需要使用GNS,因此在虚拟机环境下配dhcp和dns就稍微麻烦点了。
我是建立一个虚拟机做dhcp server,在本机物理主机(win 7)上用ISC BIND来做dns解析。
在虚拟机dhcp server的配置如下:
[root@dhcpserver sbin]# cat /etc/dhcp/dhcpd.conf ddns-update-style interim; ignore client-updates; ## DHCP for public: subnet 192.168.56.0 netmask 255.255.255.0 { default-lease-time 43200; max-lease-time 86400; option subnet-mask 255.255.255.0; option broadcast-address 192.168.56.255; option routers 192.168.56.1; option domain-name-servers 192.168.56.3; option domain-name "grid.localdomain"; pool { range 192.168.56.10 192.168.56.29; } } ## DHCP for private subnet 192.168.57.0 netmask 255.255.255.0 { default-lease-time 43200; max-lease-time 86400; option subnet-mask 255.255.0.0; option broadcast-address 192.168.57.255; pool { range 192.168.57.30 192.168.57.49; } }
在windows上的BIND配置:
C:\Windows\SysWOW64\dns\etc>cat named.conf options { directory "c:\windows\SysWOW64\dns\etc"; forwarders {8.8.8.8; 8.8.4.4;}; allow-transfer { none; }; }; logging{ channel my_log{ file "named.log" versions 3 size 2m; severity info; print-time yes; print-severity yes; print-category yes; }; category default{ my_log; }; }; ###################################### # ADD for oracle RAC SCAN, # START FROM HERE ###################################### zone "56.168.192.in-addr.arpa" IN { type master; file "C:\Windows\SysWOW64\dns\etc\56.168.192.in-addr.local"; allow-update { none; }; }; zone "localdomain" IN { type master; file "C:\Windows\SysWOW64\dns\etc\localdomain.zone"; allow-update { none; }; }; zone "grid.localdomain" IN { type forward; forward only; forwarders { 192.168.56.108 ;}; }; ###################################### # ADD for oracle RAC SCAN, # END FROM HERE ###################################### C:\Windows\SysWOW64\dns\etc>
在cluster节点的主机上,看hosts文件是这样的:
[oracle@ol6-121-rac1 ~]$ cat /etc/hosts ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 127.0.0.1 localhost.localdomain localhost # Public 192.168.56.121 ol6-121-rac1.localdomain ol6-121-rac1 192.168.56.122 ol6-121-rac2.localdomain ol6-121-rac2 # Private 192.168.57.31 ol6-121-rac1-priv.localdomain ol6-121-rac1-priv 192.168.57.32 ol6-121-rac2-priv.localdomain ol6-121-rac2-priv #Because use GNS, so vip and scanvip is provide by GNS # Virtual #192.168.56.103 ol6-121-rac1-vip.localdomain ol6-121-rac1-vip #192.168.56.104 ol6-121-rac2-vip.localdomain ol6-121-rac2-vip # SCAN #192.168.56.105 ol6-121-scan.localdomain ol6-121-scan #192.168.56.106 ol6-121-scan.localdomain ol6-121-scan #192.168.56.107 ol6-121-scan.localdomain ol6-121-scan [oracle@ol6-121-rac1 ~]$
一会等安装完成后,我们再来通过ifconfig来看看带起来的IP。
安装比较简单,基本是一路next下去了。我的环境是2个节点的cluster,安装的时候,我建立成了一个hub node,一个leaf node。这其实可以互相转换的。
安装过程我们看图说话:
注意,这里的gns的subdomains是grid.localmain,和我们之前在dhcp配置中的子域一致。
注,这里如果看不到asm的disk,change discovery path一下即可。
注,如果你之前的GNS环境没建立好,如dhcp有问题,或者dns有问题,那么在这一步执行root.sh的脚本的时候很有可能报错。
[root@ol6-121-rac1 ~]# sh /u01/app/12.1.0.1/grid/root.sh …… CRS-2672: Attempting to start 'ora.DATA.dg' on 'ol6-121-rac1' CRS-2676: Start of 'ora.DATA.dg' on 'ol6-121-rac1' succeeded 2013/07/12 15:37:01 CLSRSC-349: The Oracle Clusterware stack failed to stop <<<<<<<< Died at /u01/app/12.1.0.1/grid/crs/install/crsinstall.pm line 310. <<<<<<<<<< The command '/u01/app/12.1.0.1/grid/perl/bin/perl -I/u01/app/12.1.0.1/grid/perl/lib -I/u01/app/12.1.0.1/grid/crs/install /u01/app/12.1.0.1/grid/crs/install/rootcrs.pl ' execution failed [root@ol6-121-rac1 ~]#
查log可以看到如下,但是根据相关的报错去mos或者网上查,基本没有什么信息,(我不知道现在有没有,但是1年前我去查的时候,资料基本是空白),只有11g cluster安装的时候,有相似的报错。报错的原因是GNS的配置问题。因此,花了比较大的力气,配置了DNS和dhcp,搞好了GNS,才使得安装顺利。如果你也安装flex cluster,请一定要配置好网络配置好GNS。
log中的报错信息: …… > CRS-2794: Shutdown of Cluster Ready Services-managed resources on 'ol6-121-rac1' has failed > CRS-2675: Stop of 'ora.crsd' on 'ol6-121-rac1' failed > CRS-2799: Failed to shut down resource 'ora.crsd' on 'ol6-121-rac1' > CRS-2795: Shutdown of Oracle High Availability Services-managed resources on 'ol6-121-rac1' has failed > CRS-4687: Shutdown command has completed with errors. > CRS-4000: Command Stop failed, or completed with errors. >End Command output 2013-07-12 15:37:00: The return value of stop of CRS: 1 2013-07-12 15:37:00: Executing cmd: /u01/app/12.1.0.1/grid/bin/crsctl check crs 2013-07-12 15:37:01: Command output: > CRS-4638: Oracle High Availability Services is online > CRS-4537: Cluster Ready Services is online > CRS-4529: Cluster Synchronization Services is online > CRS-4533: Event Manager is online >End Command output 2013-07-12 15:37:01: Executing cmd: /u01/app/12.1.0.1/grid/bin/clsecho -p has -f clsrsc -m 349 2013-07-12 15:37:01: Command output: > CLSRSC-349: The Oracle Clusterware stack failed to stop >End Command output 2013-07-12 15:37:01: Executing cmd: /u01/app/12.1.0.1/grid/bin/clsecho -p has -f clsrsc -m 349 2013-07-12 15:37:01: Command output: > CLSRSC-349: The Oracle Clusterware stack failed to stop >End Command output 2013-07-12 15:37:01: CLSRSC-349: The Oracle Clusterware stack failed to stop 2013-07-12 15:37:01: ###### Begin DIE Stack Trace ###### 2013-07-12 15:37:01: Package File Line Calling 2013-07-12 15:37:01: --------------- -------------------- ---- ---------- 2013-07-12 15:37:01: 1: main rootcrs.pl 211 crsutils::dietrap 2013-07-12 15:37:01: 2: crsinstall crsinstall.pm 310 main::__ANON__ 2013-07-12 15:37:01: 3: crsinstall crsinstall.pm 219 crsinstall::CRSInstall 2013-07-12 15:37:01: 4: main rootcrs.pl 334 crsinstall::new 2013-07-12 15:37:01: ####### End DIE Stack Trace ####### 2013-07-12 15:37:01: ROOTCRS_STACK checkpoint has failed 2013-07-12 15:37:01: Running as user oracle: /u01/app/12.1.0.1/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_STACK -state FAIL 2013-07-12 15:37:01: s_run_as_user2: Running /bin/su oracle -c ' /u01/app/12.1.0.1/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_STACK -state FAIL ' 2013-07-12 15:37:01: Removing file /tmp/fileBn895c 2013-07-12 15:37:01: Successfully removed file: /tmp/fileBn895c 2013-07-12 15:37:01: /bin/su successfully executed 2013-07-12 15:37:01: Succeeded in writing the checkpoint:'ROOTCRS_STACK' with status:FAIL 2013-07-12 15:37:01: Running as user oracle: /u01/app/12.1.0.1/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_STACK -state FAIL 2013-07-12 15:37:01: s_run_as_user2: Running /bin/su oracle -c ' /u01/app/12.1.0.1/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_STACK -state FAIL ' 2013-07-12 15:37:01: Removing file /tmp/fileYx9kDX 2013-07-12 15:37:01: Successfully removed file: /tmp/fileYx9kDX 2013-07-12 15:37:01: /bin/su successfully executed 2013-07-12 15:37:01: Succeeded in writing the checkpoint:'ROOTCRS_STACK' with status:FAIL
但是如果没有问题,执行的时候如下面这样,那么恭喜你,直接进入下一步配置asm了。
--在node1上执行: [root@ol6-121-rac1 12.1.0.1]# /u01/app/oraInventory/orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@ol6-121-rac1 12.1.0.1]# [root@ol6-121-rac1 12.1.0.1]# [root@ol6-121-rac1 12.1.0.1]# /u01/app/12.1.0.1/grid/root.sh Performing root user operation for Oracle 12c The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/12.1.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/12.1.0.1/grid/crs/install/crsconfig_params 2013/08/20 00:23:03 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert 2013/08/20 00:24:00 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.drivers.acfs' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.evmd' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.mdnsd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.mdnsd' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.evmd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.gpnpd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.gipcd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.gipcd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'ol6-121-rac1' CRS-2676: Start of 'ora.diskmon' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.cssd' on 'ol6-121-rac1' succeeded ASM created and started successfully. Disk Group DATA created successfully. CRS-2672: Attempting to start 'ora.storage' on 'ol6-121-rac1' CRS-2676: Start of 'ora.storage' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.crsd' on 'ol6-121-rac1' succeeded CRS-4256: Updating the profile Successful addition of voting disk 7998f05a43964f77bfe185cd468262c4. Successfully replaced voting disk group with +DATA. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 7998f05a43964f77bfe185cd468262c4 (/dev/asm-disk1) [DATA] Located 1 voting disk(s). CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.crsd' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.crsd' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.storage' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.storage' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.drivers.acfs' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.evmd' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.asm' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.gpnpd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.evmd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.asm' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.cssd' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.gipcd' on 'ol6-121-rac1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol6-121-rac1' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.evmd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.mdnsd' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.evmd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.gpnpd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.gipcd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol6-121-rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'ol6-121-rac1' CRS-2676: Start of 'ora.diskmon' on 'ol6-121-rac1' succeeded CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'ol6-121-rac1' CRS-2676: Start of 'ora.cssd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.ctssd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.ctssd' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'ol6-121-rac1' CRS-2676: Start of 'ora.asm' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.storage' on 'ol6-121-rac1' CRS-2676: Start of 'ora.storage' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.crsd' on 'ol6-121-rac1' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-6017: Processing resource auto-start for servers: ol6-121-rac1 CRS-6016: Resource auto-start has completed for server ol6-121-rac1 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2013/08/20 00:30:20 CLSRSC-343: Successfully started Oracle clusterware stack CRS-2672: Attempting to start 'ora.net1.network' on 'ol6-121-rac1' CRS-2676: Start of 'ora.net1.network' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gns.vip' on 'ol6-121-rac1' CRS-2676: Start of 'ora.gns.vip' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gns' on 'ol6-121-rac1' CRS-2676: Start of 'ora.gns' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol6-121-rac1' CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'ol6-121-rac1' CRS-2676: Start of 'ora.asm' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.DATA.dg' on 'ol6-121-rac1' CRS-2676: Start of 'ora.DATA.dg' on 'ol6-121-rac1' succeeded CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.crsd' on 'ol6-121-rac1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.ol6-121-rac1.vip' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.gns' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.cvu' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.scan2.vip' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.scan3.vip' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.cvu' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.scan1.vip' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.DATA.dg' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.ol6-121-rac1.vip' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.scan3.vip' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.asm' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.scan2.vip' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.gns' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.gns.vip' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.gns.vip' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.ons' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.net1.network' on 'ol6-121-rac1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'ol6-121-rac1' has completed CRS-2677: Stop of 'ora.crsd' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.evmd' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.storage' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'ol6-121-rac1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.storage' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.drivers.acfs' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.evmd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'ol6-121-rac1' succeeded CRS-2677: Stop of 'ora.asm' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.cssd' on 'ol6-121-rac1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'ol6-121-rac1' CRS-2677: Stop of 'ora.gipcd' on 'ol6-121-rac1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol6-121-rac1' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.evmd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.mdnsd' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.evmd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.gpnpd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.gipcd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol6-121-rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'ol6-121-rac1' CRS-2676: Start of 'ora.diskmon' on 'ol6-121-rac1' succeeded CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'ol6-121-rac1' CRS-2676: Start of 'ora.cssd' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.ctssd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.ctssd' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'ol6-121-rac1' CRS-2676: Start of 'ora.asm' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.storage' on 'ol6-121-rac1' CRS-2676: Start of 'ora.storage' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'ol6-121-rac1' CRS-2676: Start of 'ora.crsd' on 'ol6-121-rac1' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-6017: Processing resource auto-start for servers: ol6-121-rac1 CRS-2672: Attempting to start 'ora.cvu' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.ons' on 'ol6-121-rac1' CRS-2676: Start of 'ora.cvu' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.ons' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.scan2.vip' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.scan3.vip' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.scan1.vip' on 'ol6-121-rac1' CRS-2672: Attempting to start 'ora.ol6-121-rac1.vip' on 'ol6-121-rac1' CRS-2676: Start of 'ora.scan1.vip' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'ol6-121-rac1' CRS-2676: Start of 'ora.scan3.vip' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'ol6-121-rac1' CRS-2676: Start of 'ora.scan2.vip' on 'ol6-121-rac1' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'ol6-121-rac1' CRS-2676: Start of 'ora.ol6-121-rac1.vip' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'ol6-121-rac1' succeeded CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'ol6-121-rac1' succeeded CRS-6016: Resource auto-start has completed for server ol6-121-rac1 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2013/08/20 00:37:55 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@ol6-121-rac1 12.1.0.1]# --在node2上执行: [root@ol6-121-rac2 ~]# /u01/app/oraInventory/orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@ol6-121-rac2 ~]# [root@ol6-121-rac2 ~]# [root@ol6-121-rac2 ~]# [root@ol6-121-rac2 ~]# /u01/app/12.1.0.1/grid/root.sh Performing root user operation for Oracle 12c The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/12.1.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/12.1.0.1/grid/crs/install/crsconfig_params 2013/08/20 00:38:48 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful 2013/08/20 00:39:20 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol6-121-rac2' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'ol6-121-rac2' CRS-2677: Stop of 'ora.drivers.acfs' on 'ol6-121-rac2' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol6-121-rac2' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'ol6-121-rac2' CRS-2672: Attempting to start 'ora.evmd' on 'ol6-121-rac2' CRS-2676: Start of 'ora.mdnsd' on 'ol6-121-rac2' succeeded CRS-2676: Start of 'ora.evmd' on 'ol6-121-rac2' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'ol6-121-rac2' CRS-2676: Start of 'ora.gpnpd' on 'ol6-121-rac2' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'ol6-121-rac2' CRS-2676: Start of 'ora.gipcd' on 'ol6-121-rac2' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol6-121-rac2' CRS-2676: Start of 'ora.cssdmonitor' on 'ol6-121-rac2' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'ol6-121-rac2' CRS-2672: Attempting to start 'ora.diskmon' on 'ol6-121-rac2' CRS-2676: Start of 'ora.diskmon' on 'ol6-121-rac2' succeeded CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'ol6-121-rac2' CRS-2676: Start of 'ora.cssd' on 'ol6-121-rac2' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol6-121-rac2' CRS-2672: Attempting to start 'ora.ctssd' on 'ol6-121-rac2' CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol6-121-rac2' succeeded CRS-2676: Start of 'ora.ctssd' on 'ol6-121-rac2' succeeded CRS-2672: Attempting to start 'ora.storage' on 'ol6-121-rac2' CRS-2676: Start of 'ora.storage' on 'ol6-121-rac2' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'ol6-121-rac2' CRS-2676: Start of 'ora.crsd' on 'ol6-121-rac2' succeeded CRS-6017: Processing resource auto-start for servers: ol6-121-rac2 CRS-6016: Resource auto-start has completed for server ol6-121-rac2 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2013/08/20 00:43:36 CLSRSC-343: Successfully started Oracle clusterware stack 2013/08/20 00:43:41 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@ol6-121-rac2 ~]# Broadcast message from root@ol6-121-rac2
asm的配置,不需要该什么参数,由于我们节点2是leaf node,所以不会有启动的asm instance。
等asm配置完,flex cluster也就安装完毕了。
好了,flex cluster,安装完毕,在继续安装db之前,我们用ifconfig来看看其当前的网络情况和相关资源:
node 1是hub node,上面的情况是:
[oracle@ol6-121-rac1 ~]$ ifconfig -a eth0 Link encap:Ethernet HWaddr 08:00:27:06:72:4B inet addr:192.168.56.121 Bcast:192.168.56.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe06:724b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1204 errors:0 dropped:0 overruns:0 frame:0 TX packets:1211 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:166853 (162.9 KiB) TX bytes:208611 (203.7 KiB) eth0:1 Link encap:Ethernet HWaddr 08:00:27:06:72:4B inet addr:192.168.56.108 Bcast:192.168.56.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:2 Link encap:Ethernet HWaddr 08:00:27:06:72:4B inet addr:192.168.56.12 Bcast:192.168.56.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:3 Link encap:Ethernet HWaddr 08:00:27:06:72:4B inet addr:192.168.56.14 Bcast:192.168.56.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:4 Link encap:Ethernet HWaddr 08:00:27:06:72:4B inet addr:192.168.56.13 Bcast:192.168.56.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:5 Link encap:Ethernet HWaddr 08:00:27:06:72:4B inet addr:192.168.56.11 Bcast:192.168.56.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth1 Link encap:Ethernet HWaddr 08:00:27:4D:0D:02 inet addr:192.168.57.31 Bcast:192.168.57.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe4d:d02/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1147 errors:0 dropped:0 overruns:0 frame:0 TX packets:1858 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:523883 (511.6 KiB) TX bytes:1344884 (1.2 MiB) eth1:1 Link encap:Ethernet HWaddr 08:00:27:4D:0D:02 inet addr:169.254.2.250 Bcast:169.254.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:1871 errors:0 dropped:0 overruns:0 frame:0 TX packets:1871 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2288895 (2.1 MiB) TX bytes:2288895 (2.1 MiB) [oracle@ol6-121-rac1 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....SM.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora.DATA.dg ora....up.type ONLINE ONLINE ol6-...rac1 ora....ER.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora....AF.lsnr ora....er.type ONLINE ONLINE ol6-...rac2 ora....N1.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora....N2.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora....N3.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora.asm ora.asm.type ONLINE ONLINE ol6-...rac1 ora.cvu ora.cvu.type ONLINE ONLINE ol6-...rac1 ora.gns ora.gns.type ONLINE ONLINE ol6-...rac1 ora.gns.vip ora....ip.type ONLINE ONLINE ol6-...rac1 ora....network ora....rk.type ONLINE ONLINE ol6-...rac1 ora.oc4j ora.oc4j.type OFFLINE OFFLINE ora....C1.lsnr application ONLINE ONLINE ol6-...rac1 ora....ac1.ons application ONLINE ONLINE ol6-...rac1 ora....ac1.vip ora....t1.type ONLINE ONLINE ol6-...rac1 ora.ons ora.ons.type ONLINE ONLINE ol6-...rac1 ora.proxy_advm ora....vm.type ONLINE ONLINE ol6-...rac1 ora.scan1.vip ora....ip.type ONLINE ONLINE ol6-...rac1 ora.scan2.vip ora....ip.type ONLINE ONLINE ol6-...rac1 ora.scan3.vip ora....ip.type ONLINE ONLINE ol6-...rac1 [oracle@ol6-121-rac1 ~]$ ps -ef |grep asm root 2416 2 0 22:45 ? 00:00:00 [asmWorkerThread] root 2417 2 0 22:45 ? 00:00:00 [asmWorkerThread] root 2418 2 0 22:45 ? 00:00:00 [asmWorkerThread] root 2419 2 0 22:45 ? 00:00:00 [asmWorkerThread] root 2420 2 0 22:45 ? 00:00:00 [asmWorkerThread] oracle 2746 1 0 22:46 ? 00:00:00 asm_pmon_+ASM1 oracle 2748 1 0 22:46 ? 00:00:00 asm_psp0_+ASM1 oracle 2750 1 5 22:46 ? 00:00:22 asm_vktm_+ASM1 oracle 2754 1 0 22:46 ? 00:00:00 asm_gen0_+ASM1 oracle 2756 1 0 22:46 ? 00:00:00 asm_mman_+ASM1 oracle 2760 1 0 22:46 ? 00:00:00 asm_diag_+ASM1 oracle 2762 1 0 22:46 ? 00:00:00 asm_ping_+ASM1 oracle 2764 1 0 22:46 ? 00:00:00 asm_dia0_+ASM1 oracle 2766 1 0 22:46 ? 00:00:00 asm_lmon_+ASM1 oracle 2768 1 0 22:46 ? 00:00:00 asm_lmd0_+ASM1 oracle 2770 1 0 22:46 ? 00:00:02 asm_lms0_+ASM1 oracle 2774 1 0 22:46 ? 00:00:00 asm_lmhb_+ASM1 oracle 2776 1 0 22:46 ? 00:00:00 asm_lck1_+ASM1 oracle 2778 1 0 22:46 ? 00:00:00 asm_gcr0_+ASM1 oracle 2780 1 0 22:46 ? 00:00:00 asm_dbw0_+ASM1 oracle 2782 1 0 22:46 ? 00:00:00 asm_lgwr_+ASM1 oracle 2784 1 0 22:46 ? 00:00:00 asm_ckpt_+ASM1 oracle 2786 1 0 22:46 ? 00:00:00 asm_smon_+ASM1 oracle 2788 1 0 22:46 ? 00:00:00 asm_lreg_+ASM1 oracle 2790 1 0 22:46 ? 00:00:00 asm_rbal_+ASM1 oracle 2792 1 0 22:46 ? 00:00:00 asm_gmon_+ASM1 oracle 2794 1 0 22:46 ? 00:00:00 asm_mmon_+ASM1 oracle 2796 1 0 22:46 ? 00:00:00 asm_mmnl_+ASM1 oracle 2798 1 0 22:46 ? 00:00:00 asm_lck0_+ASM1 oracle 2886 1 0 22:47 ? 00:00:00 asm_asmb_+ASM1 oracle 2888 1 0 22:47 ? 00:00:00 oracle+ASM1_asmb_+asm1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle 6606 3162 0 22:54 pts/0 00:00:00 grep asm [oracle@ol6-121-rac1 ~]$ ps -ef |grep tns root 10 2 0 22:44 ? 00:00:00 [netns] oracle 3146 1 0 22:47 ? 00:00:00 /u01/app/12.1.0.1/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit oracle 3156 1 0 22:47 ? 00:00:00 /u01/app/12.1.0.1/grid/bin/tnslsnr LISTENER_SCAN1 -no_crs_notify -inherit oracle 3187 1 0 22:47 ? 00:00:00 /u01/app/12.1.0.1/grid/bin/tnslsnr LISTENER_SCAN2 -no_crs_notify -inherit oracle 3199 1 0 22:47 ? 00:00:00 /u01/app/12.1.0.1/grid/bin/tnslsnr LISTENER_SCAN3 -no_crs_notify -inherit oracle 3237 1 0 22:47 ? 00:00:00 /u01/app/12.1.0.1/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit oracle 6608 3162 0 22:54 pts/0 00:00:00 grep tns [oracle@ol6-121-rac1 ~]$ [oracle@ol6-121-rac1 ~]$ ps -ef |grep ohas root 1162 1 0 22:45 ? 00:00:00 /bin/sh /etc/init.d/init.ohasd run root 2061 1 1 22:45 ? 00:00:09 /u01/app/12.1.0.1/grid/bin/ohasd.bin reboot oracle 6611 3162 0 22:54 pts/0 00:00:00 grep ohas [oracle@ol6-121-rac1 ~]$
node 2是leaf node,上面的情况是:
[oracle@ol6-121-rac2 ~]$ ifconfig -a eth0 Link encap:Ethernet HWaddr 08:00:27:8E:CE:20 inet addr:192.168.56.122 Bcast:192.168.56.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe8e:ce20/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1049 errors:0 dropped:0 overruns:0 frame:0 TX packets:854 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:157826 (154.1 KiB) TX bytes:135961 (132.7 KiB) eth1 Link encap:Ethernet HWaddr 08:00:27:3B:C4:0A inet addr:192.168.57.32 Bcast:192.168.57.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe3b:c40a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1886 errors:0 dropped:0 overruns:0 frame:0 TX packets:1242 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1353953 (1.2 MiB) TX bytes:543170 (530.4 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:408 (408.0 b) TX bytes:408 (408.0 b) [oracle@ol6-121-rac2 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....SM.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora.DATA.dg ora....up.type ONLINE ONLINE ol6-...rac1 ora....ER.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora....AF.lsnr ora....er.type ONLINE ONLINE ol6-...rac2 ora....N1.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora....N2.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora....N3.lsnr ora....er.type ONLINE ONLINE ol6-...rac1 ora.asm ora.asm.type ONLINE ONLINE ol6-...rac1 ora.cvu ora.cvu.type ONLINE ONLINE ol6-...rac1 ora.gns ora.gns.type ONLINE ONLINE ol6-...rac1 ora.gns.vip ora....ip.type ONLINE ONLINE ol6-...rac1 ora....network ora....rk.type ONLINE ONLINE ol6-...rac1 ora.oc4j ora.oc4j.type OFFLINE OFFLINE ora....C1.lsnr application ONLINE ONLINE ol6-...rac1 ora....ac1.ons application ONLINE ONLINE ol6-...rac1 ora....ac1.vip ora....t1.type ONLINE ONLINE ol6-...rac1 ora.ons ora.ons.type ONLINE ONLINE ol6-...rac1 ora.proxy_advm ora....vm.type ONLINE ONLINE ol6-...rac1 ora.scan1.vip ora....ip.type ONLINE ONLINE ol6-...rac1 ora.scan2.vip ora....ip.type ONLINE ONLINE ol6-...rac1 ora.scan3.vip ora....ip.type ONLINE ONLINE ol6-...rac1 [oracle@ol6-121-rac2 ~]$ ps -ef |grep asm oracle 2663 2566 0 22:55 pts/0 00:00:00 grep asm [oracle@ol6-121-rac2 ~]$ ps -ef |grep tns root 10 2 0 22:46 ? 00:00:00 [netns] oracle 2641 1 0 22:53 ? 00:00:00 /u01/app/12.1.0.1/grid/bin/tnslsnr LISTENER_LEAF -no_crs_notify -inherit oracle 2666 2566 0 22:55 pts/0 00:00:00 grep tns [oracle@ol6-121-rac2 ~]$ ps -ef |grep ohas root 1124 1 0 22:46 ? 00:00:00 /bin/sh /etc/init.d/init.ohasd run root 2058 1 2 22:47 ? 00:00:14 /u01/app/12.1.0.1/grid/bin/ohasd.bin reboot oracle 2668 2566 0 22:55 pts/0 00:00:00 grep ohas [oracle@ol6-121-rac2 ~]$