Manual Upgrade of IDM using Sun App Server with a JDBC Data Source

Hello Again .

I ran into this issue while troubleshooting a post process error doing a manual upgrade from 8.0.0.5 to 8.1 using Sun App Server with a JDBC Data Source. I will show the errors I was getting and then show you the steps on how to do this.

Note: The documentation is correct but if you don't read it you will miss a few gotchas that my customer and I ran into.  

Problem:

For the customer issue they were doing a manual upgrade from IDM 8.0.0.5 to 8.1 using  Sun App Server 9.1 with Microsoft SQL as their repository connecting through JDBC dataStore.

What they did to upgrade was:

Take a IDM.WAR from 8.0 and unzipping to a folder called IDM8.0

Then make a copy of IDM8.0 directory (files and folders) called IDM8.0.0.3

Take the IDM.war for 8.0.0.3 and unzip this into the IDM8.0.0.3 folder then perform a build.

Finally take the IDM.war created from the build and deploy this, but before deploying the code they would import the update.xml. The process was failing in the post process with the following error.

When running the post process script we see:

2009-05-04T17:06:41.862-0400 Starting Update Post-Process
2009-05-04T17:06:41.877-0400 Restoring customized files
2009-05-04T17:06:41.878-0400 Reading files from '/export/home/idm81/patches/Identity_Manager_8_1_0_0_20090224/savedFiles/changedFileList'
2009-05-04T17:06:41.879-0400   Done
2009-05-04T17:06:41.925-0400   Backing up '/export/home/idm81/WEB-INF/ServerRepository.xml' to '/export/home/idm81/patches/Identity_Manager_8_1_0_0_20090224/filesNotInstalled/WEB-INF/ServerRepository.xml'
2009-05-04T17:06:41.926-0400   Done
2009-05-04T17:06:41.926-0400   Copying '/export/home/idm81/patches/Identity_Manager_8_1_0_0_20090224/savedFiles/WEB-INF/ServerRepository.xml' to '/export/home/idm81/WEB-INF/ServerRepository.xml'
2009-05-04T17:06:41.927-0400   Done
2009-05-04T17:06:41.928-0400 Writing 0 entries to '/export/home/idm81/patches/Identity_Manager_8_1_0_0_20090224/savedFiles/notRestoredFileList'
2009-05-04T17:06:41.929-0400   Done
2009-05-04T17:06:41.930-0400 Writing 1 entries to '/export/home/idm81/patches/Identity_Manager_8_1_0_0_20090224/filesNotInstalled/notInstalledFileList'
2009-05-04T17:06:41.932-0400   Done
2009-05-04T17:06:41.932-0400 Done restoring customized files
2009-05-04T17:06:42.024-0400 ========= Importing '/export/home/idm81/sample/update.xml'
javax.naming.NoInitialContextException: Cannot instantiate class: com.sun.enterprise.naming.SerialInitContextFactory [Root exception is java.lang.ClassNotFoundException: com.sun.enterprise.naming.SerialInitContextFactory]
com.waveset.util.ConfigurationError: Failed to load JDBC DataSource 'jdbc/idmDS':
==> javax.naming.NoInitialContextException: Cannot instantiate class: com.sun.enterprise.naming.SerialInitContextFactory
        at com.waveset.util.JdbcUtil.getDataSourceObject(JdbcUtil.java:624)
        at com.waveset.repository.RelationalDataStore.setupJdbc(RelationalDataStore.java:3942)
        at com.waveset.repository.RelationalDataStore.init(RelationalDataStore.java:3886)
        at com.waveset.repository.ServerRepository.initDataStore(ServerRepository.java:1599)
        at com.waveset.repository.ServerRepository.getPrimaryDataStore(ServerRepository.java:1476)
        at com.waveset.repository.ServerRepository.getPrimaryDataStore(ServerRepository.java:1444)
        at com.waveset.repository.ServerRepository.init(ServerRepository.java:822)
        at com.waveset.repository.ServerRepository.<init>(ServerRepository.java:798)
        at com.waveset.repository.ServerRepository.getRepository(ServerRepository.java:148)
        at com.waveset.server.Server.init(Server.java:372)
        at com.waveset.server.Server.start(Server.java:334)
        at com.waveset.server.Server.getServer(Server.java:960)
        at com.waveset.server.Server.getServer(Server.java:936)
        at com.waveset.session.SessionFactory.getLoginConfigInfo(SessionFactory.java:1018)
        at com.waveset.session.SessionFactory.startupMode(SessionFactory.java:1085)
        at com.waveset.session.SessionFactory.getLoginModGrp(SessionFactory.java:1054)
        at com.waveset.session.SessionFactory.getSession(SessionFactory.java:185)
        at com.waveset.install.ImportXml.run(ImportXml.java:126)
        at com.waveset.install.ImportXml.main(ImportXml.java:83)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.waveset.util.CommandProcess.invokeMain(CommandProcess.java:228)
        at com.waveset.util.CommandProcess.launch(CommandProcess.java:178)
        at com.waveset.util.CommandProcess.run(CommandProcess.java:316)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.waveset.util.CommandProcess.main(CommandProcess.java:344)

Troubleshooting Steps:

Some steps I took to troubleshoot this were to check to make sure the user was a root type user.
Then made sure the customer was following the procedure below for the App Server gotcha.

With what I had seen this issue seemed to be with the repository and the lh command. I then
had customer run the lh command in debugging mode.

This is what I saw:

sh ./lh -D"trace.enabled=true" -D"trace.destination=STDOUT" -D"trace.level.com.waveset.repository.RelationalDataStore#getRawConnection=4" setRepo -v -tSQLServer -icom.sun.enterprise.naming.SerialInitContextFactory -fjdbc/idmDS -uiiop://localhost:19002 -Uwaveset -Pwaveset123

CLASSPATH=/apps/sjsas91/as/lib/appserv-admin.jar:/apps/sjsas91/as/lib/appserv-rt.jar:/apps/sjsas91/as/imq/lib/imq.jar: \\ /apps/sjsas91/as/lib/j2ee.jar:/apps/sjsas91/as/domains/idmanager_dev/lib/sqljdbc.jar JAVA_OPTS=-Dorg.omg.CORBA.ORBInitialPort=19002 -Dorg.omg.CORBA.ORBInitialHost=localhost

Defaulting administrator to 'configurator'.
Defaulting credentials to 'configurator'.
20090506 17:12:02.287 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:03.816 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@3945e2
20090506 17:12:06.666 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:06.666 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@bc36ff
20090506 17:12:07.206 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:07.207 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@eff545
20090506 17:12:07.241 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:07.241 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@3cfaab
20090506 17:12:07.347 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:07.347 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@5b55a9
20090506 17:12:07.365 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:07.365 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@54f169
Checking 'SqlServerDataStore:jdbc/idmDS'...
20090506 17:12:07.761 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:07.765 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@1c5a33b
20090506 17:12:07.767 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:07.769 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@145c414
20090506 17:12:07.813 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:07.816 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@14db52b
Switching to 'SqlServerDataStore:jdbc/idmDS'...
20090506 17:12:08.571 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:08.573 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@10e6cbd
20090506 17:12:08.660 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:08.662 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@1db8f8e
Getting current location....
20090506 17:12:09.025 main(0x01f04ea6) RelationalDataStore#getRawConnection() Entry no args
20090506 17:12:09.034 main(0x01f04ea6) RelationalDataStore#getRawConnection() Exit returned= com.sun.gjc.spi.jdbc30.ConnectionHolder30@1f52125
Current Location is 'SqlServerDataStore:jdbc/idmDS'
userid is 'waveset'
password is '(set)'
jdbcDriver is 'null'

This didn't give any information except that it confirmed the repo wasn't set. I then started to duplicate this and ran into the same issue . I later found out that another engineer had run into this and this is where I found out about the data source gotcha below.

_______________________________________________________________________________
TIP:

To run the lh command with debugging:

Turned on the following trace flags to the lh command

sh lh -D"trace.enabled=true" -D"trace.destination=STDOUT" -D"trace.level.com.waveset.repository.RelationalDataStore#getRawConnection=2" setRepo -tOracle -ujdbc:oracle:thin:waveset3@ORACLESERVER:3857:DBNAME -UUSERNAME -PPASSWORD

To troubleshoot the current configuration also try:

    sh lh -D"trace.enabled=true" -D"trace.destination=STDOUT" \\ -D"trace.level.com.waveset.repository.RelationalDataStore#getRawConnection=2" setRepo -c

You can get more information by changing the trace level to RelationalDataStore=2 or 4.
___________________________________________________________________________________

First Gotcha:
The gotcha I mentioned above about the Application server is the following. Basically you have to remove the j2ee.jar file from the $WSHOME/WEB-INF/lib directory and replace it. This is done before you run the setrepo command. These steps are documented in the 8.1 Install Guide Appendix D

http://docs.sfbay.sun.com/app/docs/doc/820-5594/ahtez?l=en&a=view

The steps for doing a manual upgrade on UNIX :

   1. Stop the application server and Gateway.
   2. Update the Identity Manager database. This may or may not need to be done. just depends on the version going to.
   3. Set your environment.

      export ISPATH=Path-to-Install-Software
      export WSHOME=Path-to-Identity-Manager-Installation-or-Staging Directory
      export TEMP=Path-to-Temporary-Directory

   4. Run the pre-process.

      mkdir $TEMP
      cd $TEMP
      jar -xvf $ISPATH/idm.war WEB-INF/lib/idm.jar WEB-INF/lib/idmcommon.jar \\ CLASSPATH=$TEMP/WEB-INF/lib/idm.jar:$TEMP/WEB-INF/lib/idmcommon.jar:java -classpath $CLASSPATH  \\ -Dwaveset.home=$WSHOME com.waveset.install.UpgradePreProcess

   5. Install the software.

      cd $WSHOME
      jar -xvf $ISPATH/idm.war

   6. Run the post-process.

      java -classpath $CLASSPATH -Dwaveset.home=$WSHOME com.waveset.install.UpgradePostProcess

 Note:

The upgrade post-process step runs in a separate Java virtual machine. The default heap size for this step is 1024 MB. If you experience out-of-memory exceptions during this step, set the maximum heap size value higher. To specify a custom value, set the JAVA_OPTS environment variable using the form —Xmx<heap size> where heap size is a value, such as 2048m. An example is -Xmx2048m.


Second Gotcha:

The other gotcha in this step is listed in the 8.1 Upgrade Guide. Chapter 3 Task 8 Step 7 (pg 54) .

http://docs.sfbay.sun.com/app/docs/doc/820-5594/ahtez?l=en&a=view

In here you will see a note about Data Sources. What this states is that if you are using a JDBC Data Source that is defined in your app server. The PostProcess might not work outside of the Application Server.

The fix is to temporary replace the ServerRepository.xml file that specifies the data source with another ServerRepository.xml file that specifies a JDBC DriverManager connection. Run the post Process and then restore the ServerRepository.xml file. 


   7.Change directory to $WSHOME/bin/solaris or $WSHOME/bin/linux, and set permissions on the files in the directory so that they are executable.

   8. If you installed into a staging directory, create a .war file for deployment to your application server.

   9. Remove the Identity Manager files from the application server work directory.


This fixed the issue we were seeing. Like I said before I never knew this had to be done. I spent a lot of time troubleshooting this and am thankful that another engineer pointed this to me.  This just goes to show you that even support doesn't necessarily know and or see all the issues all the time.  I hope you enjoyed this post and aren't too bored. Please let me know what topics you would like me to cover and any other comments to help make this blog better.

 - Jeff

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

I have been in support for about 10 years now and have been doing IDM support for 5 years now. I have been working for SUN for 9 years and have supported the whole JES Stack during that time.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today