X

An Oracle blog about Exadata

  • January 5, 2010

Relocating Grid Infrastructure (part 2)

Rene Kundersma
Software Engineer
In my previous post, I moved the 11gr2 Grid infrastructure (gi) home to another location. Unfortunately, as I showed, during my actions the re-run of root.sh caused the diskgroup holding the clusterware-files (vote / ocr) to be recreated.
Recreating this diskgroup would mean loosing my database data in it.
My colleagues from the development organization helped me out and told me how I could solve this. Below you can find my notes.
Please note: I am NOT telling is this a supported action. If you perform this actions, you are on your own. I can only recommend you test things first.
The current location for the gi is /u01/app/11.2.0/grid:
[root@rac1 ~]# cat /etc/oratab | grep app
+ASM1:/u01/app/11.2.0/grid:N # line added by Agent
ORCL:/u01/app/oracle/product/11.2.0/dbhome_1:N # line added by Agent

So, first let's stop all of the clusterware stack (on both nodes)
[root@rac1 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle
[root@rac1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'rac1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac1' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2'
CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2677: Stop of 'ora.orcl.db' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac1'
CRS-2677: Stop of 'ora.DATA.dg' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac1'
CRS-2673: Attempting to stop 'ora.eons' on 'rac1'
CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1'
CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded
CRS-2677: Stop of 'ora.eons' on 'rac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'rac1'
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'rac1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.

Now, create a new directory for the gi, and move gi home into that:
[root@rac1 ~]# mkdir /u01/rk
[root@rac1 ~]# mv /u01/app/11.2.0/grid /u01/rk
[root@rac1 ~]# . oraenv
ORACLE_SID = [+ASM1] ?
The Oracle base for ORACLE_HOME=/u01/rk/grid is /u01/app/oracle

Now, edit the file crsconfig_params, so that it reflects the new gi home path:
vi $ORACLE_HOME/crs/install/crsconfig_params => and change ORACLE_HOME=/u01/rk/grid

If you would run $OH/crs/install/rootcrs.pl -patch -hahome /u01/rk/grid now, you are in trouble, like I was, this is what my alert file said:
[ohasd(26079)]CRS-1339:Oracle High Availability Service aborted due to an unexpected error [Failed to initialize Oracle Local Registry]. Details at (:OHAS00106:) in /u01/rk/grid/log/rac1/ohasd/ohasd.log.

So, what was in my ohasd.log:
2010-01-05 14:48:04.516: [  OCROSD][3046704848]utopen:6m':failed in stat OCR file/disk /u01/app/11.2.0/grid/cdata/rac1.olr, errno=2, os err string=No such file or directory
2010-01-05 14:48:04.516: [ OCROSD][3046704848]utopen:7:failed to open any OCR file/disk, errno=2, os err string=No such file or directory
2010-01-05 14:48:04.516: [ OCRRAW][3046704848]proprinit: Could not open raw device

As, you can see, I forgot to change the location of my OLR, so let's do it:
vi /etc/oracle/olr.loc and change:
olrconfig_loc=/u01/rk/grid/cdata/rac1.olr
crs_home=/u01/rk/grid

After this, I ran the command again and succeeded:
[root@rac1 grid]# $OH/crs/install/rootcrs.pl -patch -hahome /u01/rk/grid
2010-01-05 14:58:28: Parsing the host name
2010-01-05 14:58:28: Checking for super user privileges
2010-01-05 14:58:28: User has super user privileges
Using configuration parameter file: crs/install/crsconfig_params
CRS-4123: Oracle High Availability Services has been started.

See the new status:
[root@rac1 grid]# crsctl status resource -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.LISTENER.lsnr
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.asm
ONLINE ONLINE rac1 Started
ONLINE ONLINE rac2 Started
ora.eons
ONLINE OFFLINE rac1
ONLINE OFFLINE rac2
ora.gsd
OFFLINE OFFLINE rac1
OFFLINE OFFLINE rac2
ora.net1.network
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.ons
ONLINE OFFLINE rac1
ONLINE OFFLINE rac2
ora.registry.acfs
ONLINE ONLINE rac1
ONLINE ONLINE rac2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac1
ora.oc4j
1 OFFLINE OFFLINE
ora.orcl.db
1 ONLINE ONLINE rac1 Open
2 ONLINE OFFLINE
ora.rac1.vip
1 ONLINE ONLINE rac1
ora.rac2.vip
1 ONLINE ONLINE rac2
ora.scan1.vip
1 ONLINE ONLINE rac1

Rene Kundersma

Oracle Technology Services, The Netherlands

Join the discussion

Comments ( 3 )
  • coskan Thursday, January 7, 2010
    Hi Rene,
    I think it is not clear Which step is the replacement of running root.sh
    Can you please clarify this
    Regards
  • rene.kundersma Thursday, January 7, 2010
    Hi,
    Sure,
    In my previous post (http://blogs.oracle.com/XPSONHA/2009/12/relocating_grid_infrastructure.html) I ran root.sh. This action is not needed anymore when you run rootcrs.pl (like I did in http://blogs.oracle.com/XPSONHA/2010/01/relocating_grid_infrastructure_1.html)
    Hope that helps
    Rene
  • Coskan Sunday, January 10, 2010
    Rene,
    Isn't the command below
    rootcrs.pl -delete -force -lastnode
    doing the dropping of the disk from configuration or removing metadata from the asm disk?
    If it is then you again need to recreate the disks am I wrong ?
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.