Monday May 05, 2008

OpenSolaris and Amazon EC2

As many of you might have got the news by now about the planet's best Operating System has been made available on one of the most scalable Compute Cloud named Amazon EC2(Elastic Compute Cloud).
This brings you the best of the both worlds which includes the Amazon Web-Services and OpenSolaris code base. If you haven't checked how cool OpenSolaris is I would suggest to get a copy of OpenSolaris and check for your self. Once you get this just get your hand dirty on xVM and try to play with the virtualization it offers. There are may more things that has been included with this OpenSolaris and some of them are as:
. Developed under Open Source community.
. New subscription support.
. Various tools needed for development eg. Sun Studio, Netbeans and the SAMP stack.
. Different virtualization technology
. Fast and easy installation with a single click and simply add software via the new Image Packaging System (IPS).

Combining Amazon EC2 on demand compute capacity with their integrated storage and running OpenSolaris on top of that gives you unparalleled env to run your application in secure, reliable and easily scalable.
OpenSolaris makes it possible to run your application on EC2 while taking or retaining the differentiating features of the Solaris Operating system and they include:

1. DTrace - You continue to do the same Dynamic Tracing over EC2.
2. Binary and Backward Compatibility - As with Solaris 10 you continue to run your application unmodified on OpenSolaris. So you can run your oldest code unmodified without breaking a thing.
3. Open Source - OpenSolaris is developed under OpenSolaris community with participation from thousands of developer worldwide. It has more predictable life cycle and support service so you can plan and upgrade your system when needed.
4. Easy to use - If you have given a try to install OpenSolaris you would know how easy it is.

So how does the whole thing work ?
Here is the what needed to be done to have your own OpenSolaris running on Amazon EC2:
You must have an account with Amazon WebSerives
Register for OpenSolaris AMI
Sun gets back to you with the details
Use the AMI/API tool to get your env.
Install/Update or simply customize your env by installing software or customizing your system.

For more details don't forget to check here.

Wednesday Apr 23, 2008

Thank you all for conversation during IMPACT 2008, Las Vegas

We would like to thank all those who stopped by and talked to us during IMPACT. It was a great conference and it was great to talk to you all about your Solaris 10 deployments and how well your deployment is running on Solaris 10.
There were few really great session and I would suggest you get a copy of some of the interesting presentation eg.
1. IBM WebSphere and multicore platforms: Performance, scalability, and best practices
2. Exploring IBM Java 5 and Java 6: New features and enhancements
3. Understanding and tuning the Java Virtual Machine for IBM WebSphere
for you reference. They contain lots of good information which you may found useful later.

Albert has already posted the slides which we used during our presentation. They can be found here:
SIG2730: WebSphere Deployment on Solaris
TSM-2447: WebSphere Performance On Solaris

There was one other session which was on DumpAnalyzer which is not available on Solaris or HP-UX I will put another blog which will list out all the equivalent tools that already available in the Java you have on your system with WebSphere.

Saturday Apr 05, 2008

Using JConsole

Using the jconsole tool
J2SE 5.0 has comprehensive monitoring and management support. Among the tools
included in the JDK download is a Java Management Extensions (JMX)-compliant monitoring
tool called jconsole. The tool uses the built-in JMX instrumentation in the Java Virtual
Machine to provide information on performance and resource consumption of running
applications. Although the tool is included in the JDK download it can also be used to monitor
and manage applications deployed with the Java 2 Platform Standard Edition Runtime
Environment 5.0 (JRE 5.0).
Jconsole can attach to any application that is started with the JMX agent. A system property
defined on the command line enables the JMX agent. Once attached jconsole can be used to
display useful information such as thread usage, memory consumption, and details about
class loading, runtime compilation, and the operating system.

Figure jconsole Summary tab
The Summary tab provides the following information
– Uptime: how long the JVM has been running
– Total compile time: the amount of time spent in runtime compilation
– Process CPU time: the total amount of CPU time consumed by the JVM

Figure jconsole Memory tab
The Memory tab shown in above Figure provides the following information.
– Current heap size: Number of Kbytes currently occupied by the heap
– Committed memory: Total amount of memory allocated for use by the heap
– Maximum heap size: Maximum number of Kbytes occupied by the heap
– Objects pending for finalization: Number of objects pending for finalization
– Garbage collector information: Information on GC, including the garbage collector
names, number of collections performed, and total time spent performing GC
A view of the memory pictured in above Figure shows two events of interest. At
02:20 a garbage collection was triggered from the Jconsole tool. At 02:50 an
OutOfMemoryError was thrown by an application.

Figure jconsole Threads tab

Figure Thread detail

The Threads tab shown in above Figure provides the following information.
– Live threads: Current number of live daemon threads plus non-daemon threads
– Peak: Highest number of live threads since JVM started
– Daemon threads: Current number of live daemon threads
– Total started: Total number of threads started since the JVM started (including
daemon, nondaemon, and terminated)
The Classes tab provides the following information.
– Current classes loaded: Number of classes currently loaded into memory

– Total classes loaded: Total number of classes loaded into memory since the JVM
started, including those subsequently unloaded
– Total classes unloaded: Number of classes unloaded from memory since the JVM

Figure jconsole VM tab
The VM tab shown in above Figure provides the following information.
– JVM type and version
– Uptime
– VM command line arguments
– Class, Library, and Boot Class paths
– JIT Compiler
– Operating System information
In addition to monitoring, jconsole can be used to dynamically change several parameters in
the running system. For example, the setting of the -verbose:gc option can be changed so
that garbage collection trace output can be dynamically enabled or disabled for a running
application. You can also perform a GC by clicking the Perform GC button on the Memory
Getting started with jconsole
1. Start the application with the option. This option sets
the system property which enabled the JMX agent.
2. Start Jconsole with the jconsole command. Jconsole ships as a binary in the
$JAVA_HOME/bin directory. (JAVA_HOME indicates the home directory of the J2SE
installation. For example, /opt/IBM/WebSphere/AppServer/java/bin)

Figure Jconsole Connect to Agent dialog box
3. When Jconsole starts it shows a window with the list of managed VMs on the machine as
shown in Figure 9-12. The process-id (pid) and command line arguments for each VM are
printed. Select a VM, click Connect, and Jconsole will attach to it.
For more information on the monitoring and management features, and details on how to use
Jconsole, read the following document.

Tuesday Mar 18, 2008

Coming to IMPACT 2008

This year IBM is having IMPACT 2008 event in Las Vegas, NV from 6 April 2008. If you are planning to be there then here are the few sessions which we are presenting you may find interesting. BOF is going to be very very interesting and we are planning to have a open discussion where we can go through whatever is most needed by customers/WAS admins for their WAS deployments on Solaris 10. If you are looking for some collaterals or white-papers which is not available today we can certainly take down your request, prioritize them and then make them available as soon as possible. So don't miss this opportunities to talk to us and let us know what are the information needed for worry free deployment of WAS on Solaris. The details of the sessions are as follows which we will be presenting :

1) Technical Session: Performance Best Practices for WebSphere Application Server Version 6.1 on Solaris 10
Session #: TSM-2447
Session Time: Thu, 10/Apr, 04:45 PM - 06:00 PM
Location: Room 118

2) BOF: Deployment Best Practices for WebSphere Application Server Version 6.1 on Solaris 10
Session #: SIG-2730A

3) IT Executive Session: Sun and IBM WebSphere Message Broker: Configuration, Deployment and tuning on Solaris 10
Session #: ITE-2695

Wednesday Feb 27, 2008

Netbeans and Clearcase: versioning support in Netbeans

Sometime back we started the early work for ClearCase plugin along with the VCS team and now this has been in BETA. I am sure people is going to like this plugin as it looks really great and it maintains the consistency across the other versioning system. Even if the ClearCase is such a complicated software it will give you really simple, easy and intuitive action menu. If you have used any other versioning plugin from NB then your experience would remain the same. I would invite you to try this plugin and send your feedback to NB Engg so future enhancement can be made available to users.
There are couple of places you can check the details and they are as:
Roumens's Blog on ClearCase plugin

And if you would like to provide feedback don't forget to visit:

Thursday Feb 21, 2008

Netbeans 6.1 plugin for WebSphere V6.0 and V6.1

Together with my colleagues we worked on the plugin for Netbeans IDE for WebSphere. It was initially designed to work for Netbeans 5.x IDE and WebSphere Application Server 6.0. We did some work to make it available with Netbeans 6.0 and it was at update center for some time but it should be part of the main binary. Recently I have been continuing this plugin work with my team member and I am happy to share that it is available as Netbeans 6.1 Milestone 1 release. The current phase of port is done and if you are interested you can download the plugin from here. The details can be found at Milestone 1 report page and WebSphere plugin screen shots are there for your reference.
The next step would be to add more functionality as you would like. Just provide us feedback at the plugin web-site.

Wednesday Jan 16, 2008

Redbook for WebSphere on Solaris

Last couple of week we don't had any update over here as I was working with some of the IBM folks(really great ones..) on a Redbook project. The redbook is titled something like "IBM WebSphere on Sun Solaris" and will be available in couple of weeks. Just wanted to let everyone know that this is going to be a onestop information for your Websphere deployment on Solaris. I am sure some of the chapters are really great and will answer may of the questions being asked frequently.

Here is the link to the internet draft:

Monday Oct 22, 2007

UltraSPARC T2 and WebSphere Application Server licensing update

Following the release of this new processor and server based on this IBM has updated the PUV(Processor Unit Value) licensing information.
I will not elaborate on the cost benefit analysis but you can make out from the data that this seems to be very cost effective server in terms of licensing cost. The details are here for your reference:
Processor Value Unit table

Tuesday Oct 09, 2007

UltraSPARC T2 and WebSphere Application Server

As Sun continue to provide breakthrough multi-core technology and systems based on that you are going to have your hands on this new cool server very soon. As you already may know that its predecessor(Sun Fire T2000) has established itself as the best J2EE App Server Platform (In fact it dominates the other area too but I will leave that to my colleagues to comments on). The details from SPEC web-site are here for your reference:
BEA WebLogic Server SPECjAppServer2004 result on Sun Fire T2000
IBM WebSphere Application Server SPECjAppServer2004 result on Sun Fire T2000
Oracle Application Server SPECjAppServer2004 result on Sun Fire T2000
Sun Java Application Server SPECjAppServer2004 result on Sun Fire T2000

Now you are going to see the next generation server named Sun SPARC Enterprise T5120/T5220 and this is going to continue to lead this segment and other as well. Don't forget to check this blog.

So coming to the point I got my hands on one of these system and did some performance study of WebSphere Application Server V 6.1 and I found these systems to be very impressive. The results of our tests shows that it can give you double the performance of T2000. Due to limitation of 4GB of address space for 32-bit process and the App Server's own scalability, I think none of the App Server will be able to scale and fully utilize this system. So I used multiple instance to maximize the system usages. As the software remains the same the tuning and tuning recommendations are not going to change.

Here is what I have done with my WebSphere Application Server to get best out of this system:

Thread Pools:
ORB: 39/39
Web Container: 47/47
Default: 20/20

EJB Cache: 60000 (previously recommended setting on my blog would be ok too).

JVM Settings:
Heap : 2300-2600
-server -XX:NewSize/-XX:MaxNewSize 1/3rd of Heap
-Xss128k -XX:LargePageSizeInBytes=256m -XX:+AggressiveOpts
-XX:+UnlockDiagnosticVMOptions -XX:-EliminateZeroing -XX:+UseParallelGC
-XX:ParallelGCThreads=16 -XX:+UseParallelOldGC

Connection Pool:
Min/Max Connection: 160/160
Statement Cache: 120

Then the usual stuff we do with TCP Connection settings, disabling the performance monitoring framework, setting the keep-alive etc.

Here are few of the setting I put in my /etc/system and which helped a lot for network performance and interrupts distribution:
set ip:ip_soft_rings_cnt = 8
set ddi_msix_alloc_limit = 8

Some of the issues I came across and you should also keep an eye on that:
Use intrstat(1M) and mpstat(1M) to find out how the interrupts processing going on. intrstat will give you the interrupts processing for different devices and which core is handling that and then you can look at the mpstat(btw this is very long so redirect to some files to take a look at it later) which will tell you if it is a bottleneck.
DB Connection – They always originates from whatever is returned by gethostbyname(3NSL) API and thus it may become bottleneck. So I used containers so the result of these API call are different i.e that will hostname(1) of your zone.
Full GC – Keep an eye on full GC and set the New Generation appropriately.

Note: If you are looking for more information on UltraSPARC T2 don't forget to check

Disclosure Statement:
SPEC, SPECjAppServer2004 are reg tm of Standard Performance Evaluation Corporation. Links are from

Friday Jun 22, 2007

Using Sun Java System Web Server V7.0 with WebSphere Application Server

As you already know that WebSphere Application Server support different version of Sun Java System Web Server.
Sometime back Version 7.0 has been released and it also works fine with different versions of WebSphere Application Server.
When you install the plugin here are the action that take place:

  • Lays out the binary
  • Installs the GSKit component
  • Generates the plugin_cfg.xml config file
  • Registers this plugin with the Sun Java System Web Server which means adding few lines to magnus.con and commenting two lines in obj.conf and adding two lines. This is what makes the Web Server during startup to load the plugin.

Here you can find the list of changes to configuration files when you upgrade from Version 6.X to Version 7.0 fi you are wondering what has been changed.
Once this is done your web-server is ready to route your request to your app server. There's a migration wizard as well if you want to migrate your previous version of web server to this version. If you don't want to re-install the plugin and get it migrated to new web-server here is a short cut for you:

  • Go to the config directory of Sun Java System Web Server 7.0 e.g /export/sun/webserver7/https-webserver7-host/config
  • Open the file maguns.conf in any editor eg vi
  • add this line to end of file: Init fn="as_init""/export/IBM/WebSphere/Plugins/config/web
    server1/plugin-cfg.xml". This can be copied from previous magnus.conf file.
  • Save and quit the editor.
  • Now open the file: obj.conf and comments these tow lines:
    NameTrans fn="ntrans-j2ee" name="j2ee"
    Error fn="error-j2ee"
  • Add the following lines right after :
    Service fn="as_handler"
    AddLog fn="as_term"

    Restart the web-server and now you should be passsing the request to the app server.

    Thursday Jun 21, 2007

    SMF and WebSphere

    During discussion with one of the customer we found that there are interests around SMF for WebSphere. Taking this opportunity my friend Albert Leigh has already posted a head start and important link to get you started.

    Basic idea of SMF has been presented and it may need cover the complex scenario which I think wouldn't be a big deal to follow as long as you know what are the activity and its dependencies are to get started.

    Specially the thing are much straightforward when you plan to do things with standalone application server. The complexity arises when you go to multiple instance scenarios and you have clustering/distribution enabled. In these cases you have to take care of order in which you start the app eg. Node Manager must be started before you try to start the app server and thing like that.
    Same goes for the Web Server which has been configured through the admin interface and eligible to get started or stopped from single point of management.

    Let us know what else would you like to be done using SMF which is not just easy enough for yourself to do it. We will try to put the script to help you out.

    Wednesday Jun 20, 2007

    Notes from IMPACT 2007

    I was at IMPACT 2007 during last week of May. Could not get to my blog to add anything after that.
    Here is my take from the conference:
    It is a great conference for WebSphere.
    I got to meet lot of customer with whom I have interacted over the phone or eamil. There were lot of good sessions and if missed some of them you can get the presentation material from the impact sessions slide web-site
    There are some of the session I like most over the other are:
    The performance presentation, 64-Bit performance then all about the data grid, XD and some of the ESB/Process Server related sessions were really good and liked all of them. You should have the copy of presentation material handy because some of them you will need quite often.
    Here are some of the additional information I would like to provide:
    For 64-bit it is available on Solaris X86-64 is available since V6.0.2 and SPARC became available starting V6.1.
    For performance reasons if you like to use just one flag for JVM tuning on your Solaris platform, I would recommend referring to my previous blog

    Wednesday Apr 18, 2007

    Class Data Sharing(CDS) with WebSphere on Solaris

    J2SE 5.0 has introduced lot of new features and one of them happens to be CDS which is supposed to help application startup(obviously making it faster to start) and use less memory(reduce memory footprint). This was available J2SE 5.0 or JDK 1.5. But when enabling this feature with WebSphere Application Server(WAS) it used to fail for following reasons:
    There was a hard set limitation on the JDK it was refusing to use this feature if there are more than 8 jar files in bootclasspath. The JDK which comes with WAS happened to have more than 8 classes and this wasn't usable.

    IBM has released a WAS fixpack which addresses this issue. You can download this JDK fixpack from here and remember you need to apply just the SDK fixpack to get this feature working for you. But in case if you would like to fix other issues which has been fixed in WAS then its good idea to get the whole fixpack which can be downloaded from WAS V6.1 Fix Pack 7 for Solaris.

    To enable your WAS installation to make use of this feature you need to have this JVM option added to you generic JVM argument:
    Which can turned off by -Xshare:off or -Xshare:auto enable class data sharing whenever possible.
    Another important thing to note here that this feature in JDK 1.5 has been made available to client VM only so you need to have "-client" option as well for genericJvmArguments. Because on ergonomics is going to start the server vm if it gets classified as server class machine.

    Thursday Apr 12, 2007

    WebSphere JVM/JDK on Solaris

    I have seen people getting confused about the version of Java Development Kit(JDK) with WebSphere Application Server (WAS) on Solaris.
    Here is the details:
    So far all the versions of WAS available on Solaris uses Sun JDK. Then you might be wondering why java -version output is something like this:

    Java(TM) 2 Runtime Environment, Standard Edition (IBM build 1.5.0_06-erdist-2006 0404 20060511)
    Java HotSpot(TM) Server VM (build 1.5.0_06-erdist-20060404, mixed mode)
    IBM Java ORB build orb50-20060511a (SR2)
    XML build XSLT4J Java 2.7.4
    XML build IBM JAXP 1.3.5
    XML build XML4J 4.4.5

    This tells us that this particular version of java is equivalent to 1.5.0_06 version of Sun's JDK. However there are pieces that has been either changed or modified some of them you can make out from above and they are:

    • ORB has been replaced with something which comes from IBM SR2 release at the same level.
    • For XML you can see various package has been replaced and I think names are quite descriptive and they have version information with them.
    • Security Frameowrk

    All of the above has been replaced by IBM implementation.

    More details you can find in your WAS/Java install dir(/opt/IBM/WebSphere/AppServer/java) and the files to look for are:

    • README_FIRST - Gives you all the information about the changes and known problems.
    • ibmorbguide.htm - Gives you ORB details.
    • securityguide.ref.htm - All the information about security implementation.

    Now the question is can this JVM is upgraded using Sun JVM - As far as I understand all the updates to JVM/JDK comes as fix packs from IBM so if you violate that then support may not be available so it's not good idea to do that.
    Of course, you are free to do whatever you want if you understand the pieces and ready to be on your own. You can replace and test with new version of or upgrade your JVM to higher version but making sure that the pieces that has been changed is incorporated following the docs I pointed out. But if you run into any problem you must figure out yourself because it is not supported and even if it works its not good idea for anything other than testing.

    Tuesday Mar 27, 2007

    GSKit 7 on Soalris 10

    Global Security Kit (GSKit) which is part of the supplement CD which comes with WebSphere Application Server(WAS) ND V6.1 and is a C API that is internal to IBM it is not available as seperate entity. There is not much distinct document exists for this API. It is the crypto-engine for many IBM software products including following
    IBM Tivoli Access Manager
    IBM HTTP Server (IHS)
    IBM MQseries

    Why do we need GSKit for App/Web Server ?
    GSKit allows an application to be SSL enabled and lets you to access SSL and TLS functions from your socket application program.
    You will need GSKit in Application Server based deployment if you intend to do the following:

    • SSL between client (browser) and IBM HTTP Server.

    • SSL between the IBM HTTP Server plug-in and WebSphere Application Server.

    • SSL between IBM HTTP Server and LDAP server.

    However when you install the HTTP server or WAS the GSKit never gets installed. This is part of the Web-Server plugin install. In other cases you have to do the install manually. There is also no uninstaller for GSKit as IBM just provides the installer.
    During the installation of GSKit it does following activity which requires that you must be a root user:

    • Installation/Registration of GSKit as solaris packages

    • Soft links in /usr/bin so you default executable search path has access to all the executable as part of the GSKit.

    • Soft links in /usr/lib so your default library search path has access to all the shared libaries of GSKit.

    None of this would be needed if the software that requires access to either the shared libraries or binaries can handle their own requirement by having their LD_LIBARY_PATH and PATH variable modified suitably, but which doesn't seems to be true in this case so it is available to everybody on the system. As you already know anything under /usr/bin is accessible to everyone on the system and then /usr/lib is included as default library serach path for evryone. This poses further restriction when you take it to zones/container where you must have the links laid out before you proceed this so you must do this in global zone before
    doing GSKit install in local zones. From local zones you cannot have write permission on these two directory. Not having the write permission will not allow you to create the soft links. When you do the GSKit install in global zone then the links are laid out for particular version of GSKit in /usr/lib and /usr/bin and then when you install in local zones all the soft links becomes valid as they will be pointing to right binaries in local zones install. Another thing to note here is that you cannot chnage the location of the install.




    « July 2016