X

An Oracle blog about Exadata

  • December 29, 2009

Relocating Grid Infrastructure

Rene Kundersma
Software Engineer
Below I will describe the actions one needs to perform when the Oracle 11gr2 Grid Infrastructure ORACLE HOME needs to be moved to a new location.


Please note:

- You will loose already registered resources (like databases) from OCR (they need to be added back again).

- Default LISTENER needs to be re-configured (re-run netca from Grid Inf. home)

- You will have downtime during the action

- Your ASM diskgroup that holds your cluster disks will be recreated !



So, please again, note the ASM diskgroup will be recreated. In case your data is there, you will loose it.


These are my steps:


1. On all nodes, but the last run the delete force command. This will stop all cluster resources and deconfigure the Oracle clusterware stack on the node.
[root@rac1 ]# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -delete -force

Output:

2009-12-29 14:52:16: Parsing the host name

2009-12-29 14:52:16: Checking for super user privileges

2009-12-29 14:52:16: User has super user privileges

Using configuration parameter file: ./crsconfig_params

PRCR-1035 : Failed to look up CRS resource ora.cluster_vip.type for 1

PRCR-1068 : Failed to query resources

Cannot communicate with crsd

PRCR-1070 : Failed to check if resource ora.gsd is registered

Cannot communicate with crsd

PRCR-1070 : Failed to check if resource ora.ons is registered

Cannot communicate with crsd

PRCR-1070 : Failed to check if resource ora.eons is registered

Cannot communicate with crsd


ACFS-9200: Supported

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4000: Command Stop failed, or completed with errors.

CRS-4544: Unable to connect to OHAS

CRS-4000: Command Stop failed, or completed with errors.

Successfully deconfigured Oracle clusterware stack on this node

2. On the last node in run command again with the "-lastnode" option.
   [root@rac2 ~]# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -delete -force -lastnode

3. On both nodes create a new directory and move the ORACLE HOME to there:
   [root@rac1-2 ~]#  mkdir /u01/rk; mv /u01/app/11.2.0/grid /u01/rk

4. On both nodes create a new symlink for JRE;
   [root@rac1-2 ~]# cd /u01/rk/grid; rm -f JRE; ln -s /u01/rk/grid/jdk/jre/ JRE

5. As oracle run clone.pl, this will change all the hard coding:
   
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/rk/grid
cd /u01/rk/grid/clone/bin; run then:
perl clone.pl ORACLE_HOME=$ORACLE_HOME

output like this:
[oracle@rac1 bin]$  perl clone.pl ORACLE_HOME=/u01/rk/grid ORACLE_BASE=/u01/app/oracle

./runInstaller -clone -waitForCompletion "ORACLE_HOME=/u01/rk/grid" "ORACLE_BASE=/u01/app/oracle" -defaultHomeName -silent -noConfig -nowait

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB. Actual 1955 MB Passed

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2009-12-29_03-09-42PM. Please wait ...Oracle Universal Installer, Version 11.2.0.1.0 Production

Copyright (C) 1999, 2009, Oracle. All rights reserved.

You can find the log of this install session at:

/u01/app/oraInventory/logs/cloneActions2009-12-29_03-09-42PM.log

.................................................................................................... 100% Done.

Installation in progress (Tuesday, December 29, 2009 3:10:15 PM EST)

......................................................................... 73% Done.

Install successful

Linking in progress (Tuesday, December 29, 2009 3:10:19 PM EST)

Link successful

Setup in progress (Tuesday, December 29, 2009 3:11:39 PM EST)

................. 100% Done.

Setup successful

End of install phases.(Tuesday, December 29, 2009 3:13:38 PM EST)

WARNING:

The following configuration scripts need to be executed as the "root" user.

/u01/rk/grid/root.sh

To execute the configuration scripts:

1. Open a terminal window

2. Log in as "root"

3. Run the scripts

Run the script on the local node first. After successful completion, you can run the script in parallel on all the other nodes.

The cloning of OraHome1 was successful.

Please check '/u01/app/oraInventory/logs/cloneActions2009-12-29_03-09-42PM.log' for more details.

6. Verify your crsconfig_params (/u01/rk/grid/crs/install/crsconfig_params) and make sure this file is available on both nodes.
7. Relink the grid infra home on both nodes
As root:


# cd /u01/rk/grid/crs/install

# perl rootcrs.pl -unlock


As the grid infrastructure for a cluster owner:


$ export ORACLE_HOME=cd /u01/rk/grid

$ cd /u01/rk/grid/bin/relink


8. On each node run root.sh, begin with node1:


cd /u01/rk/grid; ./root.sh
Check /u01/rk/grid/install/root_rac1_2009-12-29_15-37-11.log for the output of root script


run rootcrs as requested on each node:


/u01/rk/grid/perl/bin/perl -I/u01/rk/grid/perl/lib -I/u01/rk/grid/crs/install /u01/rk/grid/crs/install/rootcrs.pl


Output node1:


[root@rac1 grid]# /u01/rk/grid/perl/bin/perl -I/u01/rk/grid/perl/lib -I/u01/rk/grid/crs/install /u01/rk/grid/crs/install/rootcrs.pl

2009-12-29 16:05:51: Parsing the host name

2009-12-29 16:05:51: Checking for super user privileges

2009-12-29 16:05:51: User has super user privileges

Using configuration parameter file: /u01/rk/grid/crs/install/crsconfig_params

LOCAL ADD MODE

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Adding daemon to inittab

CRS-4123: Oracle High Availability Services has been started.

ohasd is starting

CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'

CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'

CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded

CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'

CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'

CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'rac1'

CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'

CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded

CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'

CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded

ASM created and started successfully.

DiskGroup DATA created successfully.

clscfg: -install mode specified

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

CRS-2672: Attempting to start 'ora.crsd' on 'rac1'

CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded

Successful addition of voting disk 034bbf3dcd1f4f9fbf1afa38db67caad.

Successfully replaced voting disk group with +DATA.

CRS-4266: Voting file(s) successfully replaced

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE 034bbf3dcd1f4f9fbf1afa38db67caad (/dev/sdb1) [DATA]

Located 1 voting disk(s).

CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'

CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'rac1'

CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'

CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'

CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'

CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'

CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'

CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'

CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'

CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'

CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'

CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'

CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'rac1'

CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'

CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded

CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'

CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'rac1'

CRS-2676: Start of 'ora.asm' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.crsd' on 'rac1'

CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.evmd' on 'rac1'

CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'rac1'

CRS-2676: Start of 'ora.asm' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1'

CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.registry.acfs' on 'rac1'

CRS-2676: Start of 'ora.registry.acfs' on 'rac1' succeeded

rac1 2009/12/29 16:11:02 /u01/rk/grid/cdata/rac1/backup_20091229_161102.olr

Preparing packages for installation...

cvuqdisk-1.0.7-1

Configure Oracle Grid Infrastructure for a Cluster ... succeeded


Output node 2:

[root@rac2 grid]# /u01/rk/grid/perl/bin/perl -I/u01/rk/grid/perl/lib -I/u01/rk/grid/crs/install /u01/rk/grid/crs/install/rootcrs.pl

2009-12-29 16:13:33: Parsing the host name

2009-12-29 16:13:33: Checking for super user privileges

2009-12-29 16:13:33: User has super user privileges

Using configuration parameter file: /u01/rk/grid/crs/install/crsconfig_params

LOCAL ADD MODE

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Adding daemon to inittab

CRS-4123: Oracle High Availability Services has been started.

ohasd is starting

CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac1, number 1, and is terminating

An active cluster was found during exclusive startup, restarting to join the cluster

CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'

CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'

CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'

CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'

CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'rac2'

CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'

CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded

CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'

CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac2'

CRS-2676: Start of 'ora.drivers.acfs' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'rac2'

CRS-2676: Start of 'ora.asm' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.crsd' on 'rac2'

CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.evmd' on 'rac2'

CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded

rac2 2009/12/29 16:16:42 /u01/rk/grid/cdata/rac2/backup_20091229_161642.olr

Preparing packages for installation...

cvuqdisk-1.0.7-1

Configure Oracle Grid Infrastructure for a Cluster ... succeeded


9. Finished:


[root@rac1 trace]# crsctl status resource -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.asm
ONLINE ONLINE rac1 Started
ONLINE ONLINE rac2 Started
ora.eons
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.gsd
OFFLINE OFFLINE rac1
OFFLINE OFFLINE rac2
ora.net1.network
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.ons
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.registry.acfs
ONLINE ONLINE rac1
ONLINE ONLINE rac2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac1
ora.oc4j
1 OFFLINE OFFLINE
ora.rac1.vip
1 ONLINE ONLINE rac1
ora.rac2.vip
1 ONLINE ONLINE rac2
ora.scan1.vip
1 ONLINE ONLINE rac1


Rene Kundersma
Oracle Technology Services, The Netherlands

Join the discussion

Comments ( 2 )
  • Surachart Opun Tuesday, December 29, 2009
    Thank you for good article. Nice for me in last work day this year.
    I read and tested.
    A crs/install/crsconfig_params is importance file.
    I copied crs/install/crsconfig_params.ouibak to crs/install/crsconfig_params and then modified some values, that's fine.
  • rene.kundersma Wednesday, December 30, 2009
    Hi Surachart,
    thanks for the feedback.
    Rene
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.