Tuesday Dec 04, 2012

Small script to look for Project Replication actions that have failed

Today when looking at a couple of projects on a ZFS 7320 Storage Appliance I noticed on one project that one of its replication actions had failed, as I hadn't checked the Recent Alerts log yet I was not aware of this.

I decided to write a small script to check if there were others that had failed. Nothing fancy, just a loop through all projects, look at the project's replication child and compare the values of the last_sync and last_try properties and print the result if they're not equal. (There are probably more sensible ways of doing this, but at least it involves me getting the chance to put on my headphones and doing just a little bit of coding.)

 // this script will locate failed project level replication  
 // it will look at the sync times for 'last_sync' and 'last_try'  
 // and compare these, if they deviate you should investigate.  
 // NOTE! this code is offered 'as is' Run at your own risk,  
 // it will probably work as intended, but in now way can I  
 // (or Oracle) be held responsible if your server starts behaving  
 // like a three year old kid in a candy store.. (not that mine do,  
 // they are very well behaved boys...)  
 printf('Host: %s, pool: %s\n', get('owner'),get('pool'));  
 run('cd /');  
 printf("total projects: %d\n",proj.length +'\n');  
 // just for project level replication  
     run('select '+proj[i]);  
     //get all replication actions  
     preps = list();  
         run('select ' + preps[j]);  
         last_sync = get('last_sync');  
         last_try = get('last_try');  
 //       printf("target %s\n", get('target')); //why the flip does this not get the proper name?  
         if(!( last_sync.valueOf() === last_try.valueOf())){  
             printf("sync has failed for %s %s\n", proj[i], get('target'));  
 //           printf("OK %s %s\n", proj[i], get('target'));  
         run('done'); //done with the replica action  

For a more on how to run the script, or testing it please look at my previous post.

Sample output:
Host: elb1sn01, pool: exalogic
total projects: 45
sync has failed for ACSExalogicSystem cb3a24fe-ad60-c90f-d15d-adaafd595639

Friday Nov 02, 2012

Simple Scripting for your Exalogic Storage

Getting started with some simple scripting for the 7000 series of NAS storage using VirtualBox and JavaScript.[Read More]

Tuesday Mar 29, 2011

JavaZone 2011

Phew.. It's been a while since the last time I blogged, I'll pick it up soon, I have a couple of posts planned. (In Oracle I'm doing GlassFish, WebLogic and Exalogic) In the meantime I'll help spread the word about JavaZone 2011. This year it's the 10th anniversary of what has become (in my opinion at least) the biggest and best Java conference in Scandinavia, and possibly Europe as well.

I have an abstract I'll turn in soon,  (on Java and DTrace, debugging techniques etc..) the Call For Papers is open until April 25th so please help the organizers out by submitting your abstracts as well. They are doing an absolutely fantastic conference in their spare time. 

Anyway, hope to see you in Oslo in September.

Friday Mar 19, 2010

Configure Access Control for the Bridge Service in OpenMQ 4.4

In a topology with multiple brokers bridging them to move messages from one messaging domain to another is easy to achieve as I showed in my last post.
One thing I did not show was how to create the bridges when you also want to enforce access management on destinations, in this post I will show accessmanagement and multiple connection factories in the object stores.
[Read More]

Friday Jan 22, 2010

Configuring the new broker Bridge service between an OpenMQ 4.4 HA cluster and a single node.

The new OpenMQ 4.4 release has been eagerly looked forward to by myself and some of my customers for it's ability to bridge brokers without requiring a custom application to consume messages and forward it to a different broker. In this post I'll show how to set up a HA cluster and a single node and how to get the new Bridge service to move messages from destinations in one to the other. Also shown is how to configure JDBC persistence for the brokers and which settings are necessary to set up easier JMX client connectivity for the brokers.
[Read More]

Friday Sep 11, 2009

GlassFish and JMX through a firewall

Being able to access the GlassFish JMX server through a firewall is essential when you've got big deployments that are secured by firewalls and want to perform remote monitoring and management. Thanks to Steve Esserey and other helpful persons who pointed me to a solution.

In GlassFish v2.1 patch 02 (I've also tested this on GlassFish v2ur2 patch08 and Glassfish v2.1 patch04 for good measure :) a flag has been introduced  that allow you to bind the port for the JMX server's RMI stub to a known port. (Previously this port was dynamic, which, dare I say; caused a few eyebrows to be raised with firewall admins.)

Add this to the JVM options in your domain.xml file:


Restart the domain, and pending openings in the firewall for the port given in addition to the port the JMX Agent is listening on (default is 8686) you should now be able to connect to GlassFish behind a firewall using JConsole, VisualVM or Sun GlassFish Performance Monitor

The JMX URL to connect on then becomes: service:jmx:rmi:///jndi/rmi://hostname.domain:8686/jmxrmi

VisualVM accessing GlassFish through a Firewall In  the screenshot above VisualVM is shown accessing GlassFish v2.1p2 (Actually GlassFish v2ur2 p08, but they're the same)

Also note that the port is 18686 in the screenshot above, this is just because i set my port to 18686 in the jmx-connector element in domain.xml to avoid a port conflict with a different GlassFish server on the box

<jmx-connector accept-all="false" address="" auth-realm-name="adm
in-realm" enabled="true" name="system" port="18686" protocol="rmi_jrmp" security-

Note that the MQ broker entry seen in the left hand pane is the VisualVM-MessageQueue plugin set to be released with OpenMQ 4.4. Here I've connected it to OpenMQ 4.3 which happily also works.

Normally testing this when you're not in control of the firewall takes time, the adjustments to GlassFish are done in a minute or so, but the second thing I learned today is perhaps the most important: Eat lunch with the sysadmins and operations guys every now and then! Thanks to Tommy Operations who had the openings in the firewall configured and propagated in minutes (not months which is usual in large organizations)

Thursday Aug 13, 2009

Creating templates for Application configurations

I've been asked repeatedly about how one can create a template for scripting the Application Configurations a Binding Component supports so I thought I'd whip up a little recipe on it.[Read More]

Friday Mar 13, 2009

Workaround for flickering CASA editor in NetBeans, or how to change L&F

The CASA editor for NetBeans has an issue where the scrollbars rapidly flicker on/off when running on OpenSolaris and using the GTK or Motif (Nimbus has other issues) L&F. When doing some development today I discovered a workaround. You can change the NetBeans look and feel to the good old Metal L&F

To do this on the commandline:

./netbeans --laf javax.swing.plaf.metal.MetalLookAndFeel &

To get the installed L&Fs for your platform

        UIManager.LookAndFeelInfo[] looks = UIManager.getInstalledLookAndFeels();
        for(UIManager.LookAndFeelInfo look: looks){

Wednesday Mar 11, 2009

Getting at BC Application Configurations from JMX

Earlier today I got a question from one of the developers at the client I'm with right now on how can you reuse the configuration information present in a named Application Configuration for a Binding Component.

The case here was that they didn't want the configuration in multiple places in their system, but were happy with reusing the BC's Application Configuration, and wanted to do it from an EJB or servlet, that would not communicate through a BC. The EJB/Servlet must be deployed in a server where the Application Configuration is present for a particular BC.

 This is simple with JMX:

MBeanServer mServer = ManagementFactory.getPlatformMBeanServer();
Properties props =(Properties)mServer.invoke(new ObjectName("com.sun.jbi:ServiceName=ESBConfigurationService,ComponentType=System"), "getApplicationConfiguration", new Object[]{"sun-file-binding","myApplicationConfig", "server"}, new String[]{"java.lang.String","java.lang.String", "java.lang.String"});

 As the Servlet or EJB is already running "in-process" there is no need for credentials to the MBeanServer, these are read automatically from the runtime environment.

Then it's just a trivial matter of looking up the correct MBean (the ObjectName) and invoking the correct operation. JConsole or VisualVM with the MBeans plugin is a great help here.

Monday Sep 15, 2008

OpenESB at JavaZone 2008

I'll be presenting "Real-world OpenESB, best practices and experiences" at JavaZone 08 in Oslo, Norway on Thursday. JavaZone is the largest developer conference in Scandinavia. It takes place on the 17-18 September. Hope to see many of you there! Congratulations to the local Java JUG,- JavaBIN for arranging what is in my opinion the best Java conference, year after year!

Thursday Aug 07, 2008

Non GUI OpenESB installation

This entry describes using the NetBeans/OpenESB/GlassFish installer bundle in an headless mode or how you can assemble the pieces to create your own installation of OpenESB/GlassFish[Read More]

Tuesday Jul 29, 2008

Creating a simple headless build system for OpenESB projects.

(btw: This also applies to the JBI aspects of Java CAPS 6)

OK, let's face it, NetBeans is a great tool for working with your OpenESB projects; creating, compiling, testing etc. but once your project reaches a certain size it just becomes too unwieldy to open projects, build gather all the built artifacts, deploy and so on.
What you really need is some sort of scripted, headless build. I needed one, so i decided to see what could be done with a bit of Ant wrapper magic.

Enter, Ant.

I've never been much of a fan of a lot of the build systems that are imposed by the different vendors IDEs. I've never seen why I should use an IDE to build my code. A while ago, I think it was changed with NetBeans 4.x, they scrapped the existing build system and introduced Ant. NetBeans' build system is 100% Ant based. In theory this should make it possible for me to just fire up a project's build.xml file directly from the command line for any project generated within NetBeans, OpenESB projects as well.  In practice it is, but with a few small caveats.

OpenESB introduces a lot of plug-ins for NetBeans along with their associated Ant extensions. These can be used for CLI building and deployment.

In this blog post I'll describe how to start building OpenESB projects from the command-line and how to make the "behind-the-scenes magic" that NetBeans hide from you work from the command line as well.

But, first of all; a successful build system requires a lot of structuring of code.
My team uses Subversion as the SCM for the project I'm currently involved in. Subversion best practices introduce a few things that can be troublesome when you have to take into account the possibility of developing with multiple branches, tags etc. in NetBeans. (My SCM organization thoughts might be the subject of a post at some later time..)
Anyway, Subversion has the notion of named directories for tags, branches and trunk.
So a structure such as this might be created:
Project name
-->NetBeans Project A
 -->NetBeans Project Project B

Initially we thought that an organization where our source tree was grouped on three main levels was a good idea:
Composite Applications
Service Units

This proved to be somewhat hard to work with, because of the way that NetBeans references dependent projects. If we wanted the the CA to reference a SU or an EJB (in the Java directory) we would get the tag name or trunk denomination in all sorts of project related files and paths would become unusable as soon as we introduced a tag or branch.

So we went back to the drawing board and decided to create a more module like structure.

A courser structure, but sometimes you trade flexibility for practicality.

A module contains code for a particular domain of the business problem.
Modules that require relations to other modules are resolved through the use of the svn:externals tag on the branches/tags/trunk directory. This physically copies a version of the required module into the directory of the module that requires it. At the expense of disk space we hence got rid of the branches/tags/trunk bits of all file paths.
We now have a flat structure:

-->JavaModule (svn:externals the version of the JavaModule which resides on the same level as the Module1)

---> MessageDrivenEJB

All project relations are now in a flat structure making it easy to reference and build. Checkout is also easy, you get what is required to build a module.
In addition to Ant CAs, SUs and Java projects I also created some utility modules.
I created a CommonScripts module:

A version of the CommonScripts module is referenced by the other modules (svn:externals again)
Doing a svn up(date) gives me (for a Module)
-->Java module

I wanted to avoid changing the NetBeans generated build.xml and nbproject/build-impl.xml files, and at the same time remove the need for the nbproject/private/private.properties file. This file has a lot of file paths, JDK setting for build etc. stuff in it that are not relevant for all environments that you need to build in. (Developers may have the code in different places or different OS's hence the need to create specific wrappers or stubs to substitute this.)

CommonScripts module.

This module contains all wrappers and scripts that are required to build all projects in a module. It also creates some environment specific property files that replace the use of nbproject/private/private.properties)
In my setup this module is shared among all modules, no module specifics are in it, just generic build wrappers and environment properties files.
In here I've created a file environment.properties that just sets stuff like the location of GlassFish and NetBeans.
(Note! I might be wrong but I think that you need a NetBeans distribution to build the projects, however NetBeans requires a GUI to install and run. Solution: tar up your NetBeans install, ftp to the headless build server, untar it. You just need the Jar files and their structures that ship with NetBeans and the SOA plug ins.)
Contents of the environment.properties file:



#alternatives are server,domain, <cluster-name>, <instance-name>
#see http://wiki.open-esb.java.net/Wiki.jsp?page=JbiDeployServiceAssemblyTask


I've also got a .bat file (for Windows, for Unix use what's appropriate for your shell) to set the build environment in here:

set ANT_HOME=C:\\openesb20080603\\NetBeans-6.1-build-200806030105\\java2\\ant
set caps.netbeans.home=C:\\openesb20080603\\NetBeans-6.1-build-200806030105
set netbeans.home=%caps.netbeans.home%\\platform8
set JAVA_HOME=C:\\Java\\jdk1.5.0_15
set PATH=%ANT_HOME%\\bin;%JAVA_HOME%\\bin;%PATH%
set from.commandline=true

Now then; here comes the real trick:

How to let Ant know of what's in nbproject/private/private.properties for the environment, especially if you haven't got one?
Easy, create your own!
I've created a little template file here with the relevant resolving of the properties that the build.xml/build-impl.xml files require.









the @SOMETHING@ syntax in this file is replaced with an Ant task that generates an environment specific properties file based on what's in CommonScripts/environment.properties.
Small Ant snippet for it:
        <!-- creates a property file for the environment -->
    <target name="create_property_file" if="isWindowsPlatform" depends="configOS">
        <property file="environment.properties"/>
        <delete file="build_win.properties"/>

        <filter token="CAPS_NETBEANS_HOME" value="${caps.netbeans.home}"/>
        <filter token="GLASSFISH_HOME" value="${glassfish.home}"/>
        <copy file="build_win.properties.template" tofile="build_win.properties" filtering="true"/>

I've made a template file for the different OS architectures, though it would probably be enough with just one file.

Now when doing the build, include the generated property file using the -propertyfile argument with ant.
ant -f ..\\scripts\\buildmodule.xml build.module -propertyfile build_win.properties %1 %2 %3

Notice in the line above that it refers to a scripts directory.
-->CommonScripts  (through svn:externals)
-->Java module

This directory contains the module specific build script wrappers. These start the build of the CAs, SUs in the order specified in the scripts.

The scripts directory:

The buildmodule.properties file contains references to the projects built, Ant scrips and targets called by the buildmodule.xml file
A small Ant wrapper delegates to the build.xml file for the SU, CA or Java project in the module:

    <target name="execute.ant.target">
        <echo message="Executing build on:  ${internal.path}"/>
        <echo message="Buildfile:           ${internal.buildfile}"/>
        <echo message="Buildtarget:         ${internal.buildtarget}"/>

        <ant antfile="${internal.buildfile}" target="${internal.buildtarget}" dir="${internal.path}"/>


    <target name="build.module">
        <antcall target="execute.ant.target">
            <param name="internal.path" value="${EJB.path}"/>
            <param name="internal.buildfile" value="${EJB.buildfile}"/>
            <param name="internal.buildtarget" value="${EJB.buildtarget}"/>

        <antcall target="execute.ant.target">
            <param name="internal.path" value="${SU.path}"/>
            <param name="internal.buildfile" value="${SU.buildfile}"/>
            <param name="internal.buildtarget" value="${SU.buildtarget}"/>

        <antcall target="execute.ant.target">
            <param name="internal.path" value="${CA.path}"/>
            <param name="internal.buildfile" value="${CA.buildfile}"/>
            <param name="internal.buildtarget" value="${CA.buildtarget}"/>

Now, in order to do a full build traverse to the CommonScripts directory. Set the environment for Ant.
And do:
ant -f ..\\scripts\\buildmodule.xml build.module -propertyfile build_win.properties

Cleanup is also easy:
ant -f ..\\scripts\\buildmodule.xml build.module -propertyfile build_win.properties -Dinternal.buildtarget=clean
Setting the -D switch on the command line overrides what Ant loads, so it assigns ${internal.buildtarget} the value "clean"

Deployment follows the exact same pattern, but just calls a thin wrapper around the $glassfish.home/bin/asadmin tool invoking the "deploy.jbi.service.assembly" command for service assemblies.


This blog will be about software that i work with; Java, OpenESB, GlassFish and perhaps a bit about photography.


« April 2014