Friday Dec 26, 2014

JSSE and WebLogic Server in WebLogic Server 11g

 Recently, several customers want to migrate to JSSE, because they would like to have stronger cipher suites.

If you have WebLogic Server 10.3.2 or earlier versions, you will need to upgrade your Weblogic Server to the latest 11g.
There is no supported way to enable JSSE in these versions.


To enable JSSE in weblogic server 10.3.3 and later, you can add a property to JAVA_OPTIONS as described below:

    -Dweblogic.security.SSL.enableJSSE=true (WLS 10.3.3)
http://docs.oracle.com/cd/E14571_01/web.1111/e13707/ssl.htm#BABIJEJD
    -Dweblogic.ssl.JSSEEnabled=true (WLS 10.3.4 and later)
http://docs.oracle.com/cd/E21764_01/web.1111/e13707/ssl.htm#BABIJEJD

Additional Details here:
http://docs.oracle.com/cd/E14571_01/web.1111/e13707/ssl.htm#SECMG494

Please notice that as this is a new feature, there are several recommended patches for known issues:
For WebLogic Server 10.3.3:
SHA2 Certificate Throws Log Message "Ignoring The Trusted CA Certificate" (Doc ID 1538488.1)

For WebLogic Server 10.3.5 and 10.3.6 make sure you are using the latest PSU delivered:
e.g:
Oracle WebLogic Server Patch Set Update 10.3.5.0.7 Fixed Bugs List (Doc ID 1544306.1)
Oracle WebLogic Server Patch Set Update 10.3.6.0.9 Fixed Bugs List (Doc ID 1935048.1)   

Enjoy! 

Tuesday Dec 02, 2014

Why does my Weblogic Server takes a Long Time to Start?

Here are some possible causes:


1. If you are in Linux/Solaris Operating System, you can take a look to the document below:

   How to diagnose a Linux entropy issue on WebLogic Server instances? (Doc ID 1574979.1)

   Adding -Djava.security.egd=file:///dev/urandom or -Djava.security.egd=file:/dev/./urandom to JAVA_OPTIONS should help you to solve the issue.

   Low entropy can affect WLS, install an other processes, references:
   Domain Creation Hung at 70% on Linux/Solaris Machines (Doc ID 1378733.1)

2. If you are in a Solaris SPARC platform, there is a hardware limitation on this architecture, you can review the document below:

   Slow startup of WebLogic Servers on SPARC Series: T1000, T2000, T5xx0 and T3-* (Doc ID 1456488.1)    

3. Huge/complex deployments might contribute to long start-up

4. If none of the above options help, you can try collecting thread dumps e.g:

   kill -3 <PID>
   wait 10 secs
   kill -3 <PID>
   wait 10 secs
   kill -3 <PID>

   This will help us to understand why start  up is slow.
   Additional information here:
   https://blogs.oracle.com/LuzMestre/entry/why_do_i_need_to

Enjoy!

Tuesday Nov 11, 2014

How can I fix java.lang.OutOfMemoryError?

OutOfMemoryError (OOM) is a common issue that we can see in any WebLogic Server environment.

Here are some guidelines you can follow to troubleshoot the issue:

1. Have you tried increasing the heap size?


   This is the first thing you need to try to tune your environment, the issue will probably be fixed.
   Here is a nice Document that explains memory management for hotspot JDK (it mostly apply to recent versions):

http://www.oracle.com/technetwork/java/javase/memorymanagement-whitepaper-150215.pdf


2. Have you tried  the latest JDK of your version?


   Even you are using a certified JDK, older versions have several bugs that have been found along the years.
   These bugs can be related to GC algorithms, so trying to tune your JVM in an old version, you might not get the expected result.
   Also, the latest releases include several security fixes you might wnat to have in your environment.
   You can find all Oracle Java releases (including JRockit) here:
   All Java SE Downloads on MOS (Doc ID 1439822.1)

3. Are you using 32 or 64  bit JVM ?


   If your environment is certified with 64 bit JVM, always consider using 64 bit JVM for scalability.

4. What JVM and Operating System are you using?  


   Depending on if you are using Hotspot/JRockit/ other JDK  and also depending on your OS you can define the way you collect, gclogs, heapdumps, histograms.
   FAQ: JRockit and java.lang.OutOfMemoryError related issues (Doc ID 1290251.1)
   How to Troubleshoot Java Heap Memory Leak - "java.lang.OutOfMemoryError: Java heap space" (Doc ID 1236373.1)
   How to collect a heap dump?  -   https://blogs.oracle.com/LuzMestre/entry/how_to_collect_a_heap

5. I getting stuck threads and OutOfMemoryError, What to troubleshoot first?

  You need to troubleshoot the OOM first. Frequent GC can cause huge GC pauses, then stuck threads can occur.
   Resources for Garbage Collection Performance Tuning (Doc ID 1572109.1)

6. I am getting a JVM crash and OutOfMemoryError, What to troubleshoot first?

   You need to troubleshoot the OOM. After an OOM, the JVM is unstable, so a JVM crash can occur.
   Oracle WebLogic Server Support Pattern: Troubleshooting Out of Memory and Memory Leak Problems (Doc ID 877172.1)

   Enjoy! 

Wednesday Oct 08, 2014

Error Correction Policy and Lifetime Support for Weblogic Server

When you have a support contract you might want to know if you can request new patches.
Then, you might be interested in reading this doc.
Error Correction Support Dates for Oracle WebLogic Server (Doc ID 950131.1)

The document below explains the support stage you might have e.g: Premier Support, or  Extended
Support, and what are your support dates.


Enjoy!

Friday Sep 19, 2014

How to collect a heap dump?

Here are the most common ways to do it. You will use these if you have OutOfMemoryException (OOM) and you would like to take a look to the issue to determine the root cause.

1. For Hotspot/Oracle JDK, you need to add the following parameters to JAVA_OPTIONS

-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=<path>

2. Similarly for JRockit, you need to the JAVA_OPTIONS below:

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=C:\myApp\hprof-dumps

But, lets say that you are experiencing OOM, but you still don't determine the root cause of the issue.
You might probably need to restart as soon as you see performance issues or even better daily.
Then, #1 and #2 won't help.

You can use a tool like the ones below:
3. Jmap, e.g.:

<JAVA_HOME>/bin/jmap -dump:format=b,file=c:\temp\heap.hprof <PID>


4. Or if you have already a core file and you would like to have a heap dump as well, you can also get a heap dump:

jmap  -dump:format=b,file=heap.hprof $JAVA_HOME/bin/java /home/lmestre/core.27436

5. Or even jrcmd for JRockit:
%JROCKIT_HOME%\bin\jrcmd <pid of target JVM> hprofdump filename=<full path name to the target dump file name>

6. If you have a Windows Service, you can check This My Oracle Support Doc:
How Do I Get A Java Heap Dump When Running Weblogic As A Windows Service? (Hotspot JVM) (Doc ID 1264826.1)

To analyse the heap dump, you can use Eclipse MAT.
Here are some helpful documents to trobleshoot OOM:

Oracle WebLogic Server Support Pattern: Troubleshooting Out of Memory and Memory Leak Problems (Doc ID 877172.1)
ORACLE JRockit and Memory Consumption - Memory Leaks In Java Heap (Doc ID 1137239.1)
Enjoy!

Monday Sep 01, 2014

Where can I get the latest version of Java 6?

You can download the latest public versions here:

"Oracle offers updates to Java 6 only for customers who have purchased Java support or have Oracle products that require Java 6."
References:

Non public versions of all Oracle Java releases, including Oracle JDK (a.k.a Hotspot) and JRockit, here:
All Java SE Downloads on MOS (Doc ID 1439822.1)
Enjoy!




Tuesday Aug 19, 2014

How to start WebLogic Server using default scripts?

There are a few common issues reported when starting weblogic server using scripts.
1. User is not able to access weblogic console.
2. After a few days/hours weblogic server stops abruptly.
3. When user closes putty, they are not able to connect to weblogic server anymore.
4. When user closes windows command prompt, they are not able to connect to weblogic server anymore.
5. Weblogic is started using startManagedWebLogic.cmd/startManagedWebLogic.sh.


By default, WebLogic Server does not run in background mode, so after you close the window the process finishes as well.

In Linux/Unix based platforms, you need to use:

nohup ./startManagedWebLogic.sh <Server> <URL> &

In Windows platforms, you need to start Managed Servers using Windows Services:

How to Install MS Windows Services For FMW 11g WebLogic Domain Admin and Managed Servers (Doc ID 1060058.1)

http://docs.oracle.com/cd/E23943_01/web.1111/e13708/winservice.htm


There a few more reasons that could cause similar symptoms, like JVM crash, signals sent by the Operating System, and many other reasons. 
But the above steps is the first one to start.

Enjoy!

Tuesday Aug 05, 2014

How do I specify certicom ciphersuite in weblogic 11g?

I recently have a customer that was getting the error below when trying to specify certicom ciphersuite:

<SEVERE> <Fatal error in node manager server>
java.lang.IllegalArgumentException: SSL_RSA_WITH_RC4_128_MD5 Unsupported.
        at com.certicom.tls.interfaceimpl.TLSSystem.setEnabledCipherSuites(Unknown Source)
        at javax.net.ssl.impl.SSLServerSocketImpl.setEnabledCipherSuites(Unknown Source)
        at weblogic.nodemanager.server.SSLListener.init(SSLListener.java:80)
        at weblogic.nodemanager.server.NMServer.start(NMServer.java:206)
        at weblogic.nodemanager.server.NMServer.main(NMServer.java:377)
        at weblogic.NodeManager.main(NodeManager.java:31) 
Ciphersuites were not picked when setting the ciphersuites using config.xml

<ciphersuite>TLS_RSA_WITH_RC4_128_SHA</ciphersuite>
<ciphersuite>TLS_RSA_WITH_RC4_128_MD5</ciphersuite>

The error was gone setting the arguments in JAVA_OPTIONS
-Dweblogic.security.SSL.Ciphersuites=SSL_RSA_WITH_RC4_128_MD5,TLS_RSA_WITH_RC4_128_MD5

The above works for JSSE as well.

I hope this helps.
Enjoy!

Wednesday Jul 30, 2014

Where can I Download Weblogic Server?

The answer is very simple, please go the the link below and select the version you want to install.

http://www.oracle.com/technetwork/middleware/weblogic/downloads/wls-main-097127.html

It is a different story if you are adding one instance of your legacy systems, those versions are not available in the link above. You probably have the installer already.

However, if you don't have it, you can request the media as described in this My Oracle Support document: 

Requesting Physical Shipment or Download URL for Software Media (Doc ID 1071023.1)

** you need to be logged to your My Oracle Support account to access the link. 

Enjoy! 

Tuesday Jul 08, 2014

How to install Oracle JDK 7 in Solaris SPARC?

You might say that this could be unrelated but as WebLogic Server is supported in Solaris, I though it was a nice post for the Solaris users.

If you want to install Oracle JDK in most of the platforms, you just need to download the installer, un-compress it and run it.

The answer might be tricky for some of us not used to Solaris SPARC installers.

1. Download your JDK here: 

http://www.oracle.com/technetwork/es/java/javase/downloads/index.html

e.g: download: 

jdk-7u60-solaris-sparc.tar.gz
jdk-7u60-solaris-sparcv9.tar.gz

 2. Just copy the 2 files to one folder:

e.g:
jdk-7u60-solaris-sparc.tar.gz
jdk-7u60-solaris-sparcv9.tar.gz

3. un compress both files in the same folder

tar fxvz jdk-7u60-solaris-sparc.tar.gz
tar fxvz jdk-7u60-solaris-sparcv9.tar.gz

4. change to the directory

cd jdk1.7.0_60/bin

5. Enjoy your new Oracle JDK

./java -d64 -version  

That will do the trick!

Remember, if possible try using the latest JDK update you have available, so you can take advantage of the bugs and security fixes that are included in the latest releases.

References:

http://docs.oracle.com/javase/7/docs/webnotes/install/solaris/solaris-jdk.html#install-archive

 Enjoy!

Wednesday Jun 25, 2014

How to fix weblogic.socket.MaxMessageSizeExceededException when using weblogic standalone client?

MaxMessageSizeExceededException might be a known issue in weblogic server, but sometimes it can be tricky.

If you are using a weblogic standalone client like wlthint3client.jar you might hit MaxMessageSizeExceededException.

If you still see the exact error after setting MaxMessageSize in the Administration Console in Protocols->General, you might be hitting this issue.

weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '10000080' bytes exceeds
 the configured maximum of: '10000000' bytes for protocol: 't3'

Try  setting the value programatically in your java code e.g:

System.setProperty("weblogic.MaxMessageSize", "300000000"); 


You can also set the value in your JAVA_OPTIONS -Dweblogic.MaxMessageSize=300000000

References
http://docs.oracle.com/cd/E29542_01/web.1111/e13814/jmstuning.htm


Sometimes you still can see the same error after set MaxMessageSize in the Administration Console in Protocols->General, but you see the parameter being picked:

weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '30000080' bytes exceeds the 
configured maximum of: '30000000' bytes for protocol: 't3'

In these cases, you need to investigate why you have these huge messages along your environment. You can review a tcpdump to research further.

Enjoy!





Thursday Jun 05, 2014

Where are the values of java.library.path being set?

This one could be a WebLogic Server question, but this post is general for any java environment.

We were getting  at the very beginning  java library path something like this:

/home/lmestre/jdk1.6/jre/lib/amd64/server:/home/lmestre/jdk1.6/jre/lib/amd64:/home/lmestre/jdk1.6/jre/../lib/amd64

So, the question was:


Where WebLogic Server is setting java.library.path?

I never found the answer, so why don't we try to try to answer

Where the JVM is setting java.library.path?

public class LibraryPathPrinter {

   public static void main(String[] args) {
       String javaLibraryPath= System.getProperty("java.library.path");
       System.out.println("java.library.path "+javaLibraryPath );
   }

}

after a simple

javac LibraryPathPrinter.java

and then an easy

java LibraryPathPrinter

voila!

The program printed something like this

java.library.path  /home/lmestre/jdk1.6/jre/lib/amd64/server:/home/lmestre/jdk1.6/jre/lib/amd64:/home/lmestre/jdk1.6/jre/../lib/amd64

So the JVM was the culprit.
Enjoy!

Monday May 26, 2014

How to create a JMS durable subscriber in WebLogic Server?

WebLogic Server Provides a set of examples that are very helpful to get started with Weblogic Server

Here you can check how to install the examples:
http://docs.oracle.com/cd/E23943_01/doc.1111/e14142/prepare.htm


After you have installed the examples, you can find the example you want to review, in this case TopicReceive, here:

wlserver_10.3/samples/server/examples/src/examples/jms/topic

To review details of the specific example, you can open:

wlserver_10.3/samples/server/examples/src/examples/jms/topic/instructions.html

To create a Durable Subscriber, you can just set the client ID  and invoke createDurableSubscriber instead of calling createSubscriber, i.e.:

   tconFactory = (TopicConnectionFactory)
      PortableRemoteObject.narrow(ctx.lookup(JMS_FACTORY),
                                  TopicConnectionFactory.class);
    tcon = tconFactory.createTopicConnection();
    //Set Client ID for this Durable Subscriber
    tcon.setClientID("GT2");
    tsession = tcon.createTopicSession(false, Session.AUTO_ACKNOWLEDGE);
    topic = (Topic)
      PortableRemoteObject.narrow(ctx.lookup(topicName),
                                  Topic.class);
    // Create Durable Subscription
    tsubscriber = tsession.createDurableSubscriber(topic, "Test");
    tsubscriber.setMessageListener(this);
    tcon.start();



Enjoy!  

You can read more about this here:
http://docs.oracle.com/cd/E23943_01/web.1111/e13727/advpubsub.htm#CHDEBABC
http://docs.oracle.com/cd/E23943_01/web.1111/e13727/manage_apps.htm#i1097671    
http://docs.oracle.com/cd/E23943_01/apirefs.1111/e13943/WebLogic.Messaging.ISession.CreateDurableSubscriber_overload_2.html

Tuesday May 06, 2014

How to delete a JDBC Store?

If you see the message "Skipping unexpected null message encountered on store boot", you will need to recreate your JDBC store, because it got corrupted.

You will see a message like this in the weblogic server log:
<Warning> <Messaging> <BEA-282013> <Skipping unexpected null message encountered on store boot, [ sequence=4862 queue=12 name=MyModule!MyQueue state=0 deliveryCount=2 ]>

Steps to fix the issue:
1. Make sure managed server is not running

2.  Delete the corrupted JDBC store because messages are corrupted.

See Managing JDBC Store Tables

http://docs.oracle.com/cd/E23943_01/web.1111/e13701/store.htm#i1143431

e.g.:
If you have provided the prefix MY when you created your JDBC store, your table is named MYWLStore, otherwise, it is just named WLStore

To drop the table, you can use the option below or simply drop it using a database tool:

$ echo "drop MYWLStore;" > drop.ddl
$ cd <YourDomain>/bin
$. ./setDomainEnv.sh
  **Notice the . before ./
$ java utils.Schema jdbc:weblogic:oracle:DEMO weblogic.jdbc.oci.Driver -u user1 -p foobar \
  -verbose drop.ddl

3. Start managed server. The store table will be recreated when managed server is started

Enjoy!

Monday Apr 21, 2014

How to monitor JMS Queue counts using JConsole ?

The most common way to monitor your JMS resources is by using weblogic console, e.g.:

http://docs.oracle.com/cd/E23943_01/apirefs.1111/e13952/taskhelp/jms_servers/MonitorJMSServers.html

You can also use WLST

http://docs.oracle.com/cd/E23943_01/web.1111/e13738/wlst.htm#i1047470

If you like JConsole, here is the trick:


1. Set your environment
   (Linux) . ./setDomainEnv.sh
   (Windows) setDomainEnv.cmd


2. Invoke jconsole using the arguments below    

jconsole -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -J-Dcom.sun.tools.jconsole.mbeans.keyPropertyList=Location,type,Type,j2eeType,name,Name -debug


3. See messages counts as follows:


Related to the JMSServer
JMSServerRuntime > <MyJMSServer> > <MyWebLogicServer>  > Attributes > MessagesCurrentCount


Related to the a JMS Queue:
JMSDestinationRuntime > <MyQueueJNDIName>  >MyWebLogicServer > MyJMSServer > Attributes


References:
https://blogs.oracle.com/WebLogicServer/entry/managing_weblogic_servers_with


About


My name is Luz Mestre. I work as Principal Technical Support Engineer at Oracle Support. I'll post here the most interesting challenges I have in my daily work. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Search

Categories
Archives
« April 2015
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
  
       
Today