Friday Apr 19, 2013

Lions and Tigers and RangeSize, Oh My!

One of the most common mistakes I see made in ADF Business Components based applications is a failure to tune the View Objects, and specifically to tune the in Batches of parameter in the VO tuning section. This setting defaults to 1, which does not always meet the needs of the consuming UI or service interface and should generally be changed.  This is a topic that I and others have covered before.

Now certainly the batch fetch size is a crucial tuning parameter as it controls how many times the framework has to hit the database to get the rows needed to service a particular function.  However, in this posting I wanted to turn my attention to another factor which can also have a significant effect - the iterator RangeSize.

The background to this was some recent detective work on an application where the time taken to display one particular screen was suspiciously long. 
The page in question had a tabular display of data, but an inspection of the VO tuning parameters showed that a reasonable Batch size of 51 was being used. What's more the As Needed switch rather than the All at Once option in the VO tuning was being used. So the developer had done totally the right things there.
Running a SQL Trace on the page revealed an interesting thing though.  Because the batch size was pretty high we'd expect that the framework would have to only do one or at most two fetches from the cursor to satisfy the needs of that table.  However The TKProf output showed that in fact over 150 fetches took place retrieving over 8000 rows!

My thought processes in diagnosing this were to look in the following places:
  1. Are there alternative VO Instances defined on the AM where the tuning parameters are different (e.g. ALL_ROWS was specified)? We know the base definition is OK but it could be overridden.  
  2. Any programmatic calls to change the Batch Size or fetch node in the VO?
  3. Any programatic calls to last() on the rowset or iterations through the rowset? 
  4. Check for a RangeSize of -1 on the iterator definition in the pageDef files.

All of these drew a blank.  The last one in particular felt like the problem but a search for the value of -1 in the pageDefs of the UI project only turned up legitimate usages of the -1 value. 

Hold on I don't Understand This RangeSize?

Maybe I should take a quick step back and explain the iterator RangeSize.  So, as we've seen, the tuning options in the view Object will control how often the VO has to go back to the database to get a specific number of rows. The iterator rangeSize is defined in the pageDef file for a particular page, fragment or method activity and it defines how many rows the UI layer should ask the service layer (in this case the VO) for. 
Here's a typical definition that you'll see in the pageDef:
<iterator Binds="EmployeesView1" 
          RangeSize="25" 
          DataControl="HRServiceAMDataControl" 
          id="EmployeesView1Iterator"
          ChangeEventPolicy="ppr"/>

You'll see that the rangeSize here is set to 25 which just happens to be the value that is defaulted in when you drag and drop a binding into the page.  However, it turns out that 25 is not the default value, something which has a bearing later in this investigation as we'll see.
So in this default case when the iterator is asked for data, it in turn will ask the VO for 25 rows, and if the VO does not already have that many rows in the cache it will have to go back to the database as many times, as determined by the batch-size, as it needs to get enough data.

Back to the Problem page 


As it happens the pageDef for the table displaying the problem VO was indeed the defacto default of 25, so, sad to say, it was not the obvious suspect at fault, more investigation was needed.

At this stage the investigation splits into a couple of parallel efforts, manual code inspection, and tracing using the ADF Logging capability to try and work out what interactions were happening with the problem VO.

Welcome to the Internet, Please Tread with Care

What can we trust? Well in the world of ADF Blogs in the wild there are some great bloggers, but that does not mean that you can just copy without thinking.  It turned out that one of the problems with this application was to fall foul of copy-without-thinking syndrome.
The code in question seems innocent enough, it's published out there on the internet as a way of refreshing an iterator:

//Please, Please Don't do this! (My comment)
DCIteratorBinding iterBind= (DCIteratorBinding)dc.get("<your iterator name>");  
iterBind.refresh(DCIteratorBinding.RANGESIZE_UNLIMITED);  

Two facts to discuss in relation to this code snippet:
  1. Read the JavaDoc for DCIteratorBinding - the refresh() method is very clearly marked as Internal only, applications should not use. That's what we call "a hint" in the trade, you can choose to ignore it but don't come crying...
  2. Look at that parameter being passed to the refresh method DCIteratorBinding.RANGESIZE_UNLIMITED - can you guess what that does? Yes it does the same as setting the RangeSize to -1 and will cause the VO to be asked for all of the data for that query. You can see how bad that could be if the VO has the potential to return a lot of rows.
So something to put right already. 

But Wait, There's more!

Although the call to refresh was a great catch and the application will be better without it, it turned out not to be the cause - darn. 
However, the parellel effort to run some more tracing found the smoking gun.  The ADF Log trace showed a double execute on the iterator for the VO in question, or to be more precise, executes on two different iterators bound to the same VO from different regions on the page. 
A useful diagnostic here was then to set a breakpoint in the setRangeSize() of oracle.adf.model.binding.DCIteratorBinding.  Doing this we could see that the first iterator execution was actually responsible for setting the RangeSize to -1 and the second to the value we where expecting for that table based on the pageDef. 
All credit to the development team I was working with here who ferreted out the actual problem, it was finally down to one of omission. 

Recall I made the statement earlier about 25 being the defacto default for the RangeSize? Very true, when you create a binding, that's what the IDE puts in for you.  But what's the actual default? Well that turns out to be -1.  So if you omit the RangeSize from the iterator definition by intent, or mistake, you're going to have a potential side effect you may not have expected!  That was exactly the problem in this case  - no RangeSize. 

Specifically the problem was caused by a method binding in one of the bounded task flows in the page. The pageDef for this method activity included and iterator defined for the problem VO but without a RangeSize defined.  

Lessons to Learn

  1. For each peice of UI that is bound to data understand how the user is going to use that screen - will they view just one record, will they scroll through several or a lot, will they only scroll downwards and process one record at a time and so on. You can't effectivly tune a page until you understand how it will be used.
  2. Always tune your VO definitions or instances to the requirements of each endpoint.  If this means having multiple VO definitions or multiple VO instances with different tuning parameters then do it.
  3. If you have consciously set your Batch size in the VO tuning parameters to a small number because that's all you see on the page at once then also tune the pageDef iterator RangeSize. Otherwise that defacto 25 RangeSize could generate a bunch of unnecessary calls back to the database. Think about it, if you have a single record form and you've not tuned the VO Batch size and you've not tuned the iterator RangeSize, you could be doing 25 roundtrips to the database just to see one record.  Great way to please your DBA!
  4. Use a RangeSize of -1 with caution.  It has its place for iterators that serve short lists used for lookups and menus but anything else is an exception.
  5. Don't blindly copy everything you see in the blog-o-sphere. If you don't recognize an API call then look it up.  If something says it's for internal use only, then guess what, don't use it.
  6. Never ever, define an iterator in your pageDef without an explicit rangeSize. If you need to see all rows, say so with the -1 value, otherwise use a positive integer.

We all go home older and wiser...


Friday Jul 20, 2012

The ImageMap Pattern

In my last article, I alluded to the fact that the associated sample combined a bunch of existing patterns and techniques, and that I would progressively write those up.  In this article I'm going to talk about the first of these, the ImageMap pattern.

What's the Use Case?

 This pattern is all about solving a pretty common problem, I have, in my service model, some kind of codified value (e.g. a Type or a Status) which I want to reflect visually in my UI using an image. Now if you only have a couple of options to represent then you can use a simple ternary expression in the page, switching based on the value you're getting from the service.  However, once you get over three or four options the EL starts to get hard to read and maintain, and face it, sometimes you might not even notice that it's evaluating to the wrong thing if the expression is wrong.  So what are the options for approaching this? 

  1. Do the translation in the service layer and provide an attribute containing the correct image to display. 
  2. Manage some simple lookup into a shared UI level resource

Option (1) just feels wrong because you'd be letting client side information, the names of image files in this case,  leak into the wrong layer, so the shared resource approach really looks like the way to go. 

Implementation

The approach that we take in this pattern is to exploit the ability of Expression Language to be able to refer to both arrays and maps using the "[<value>]" syntax. For example the expression mybean.image[1] can actually mean several things depending on what "image" is in this case. If the getImage() method in the underlying mybean returns a List then this would translate to pull out index 1 from that list. If on the other hand getImage() returns a Map then the get() method will be called on the map with a key of (in this case) "1".


We can exploit this behavior, and particularly the understanding of the Map expressions to define a mapping between a piece of data from the model, such as a status code, and a particular image to use to represent that. To illustrate this let's take a simple example where we have some possible string status values in the datamodel and want to map that into different images, thus:

Code from the Service Image To Use
 TABLE  /images/table.png
 VIEW  /images/view.png
FUNCTION  /images/plsql_func.png
 ...  ..

Notice that the names of the images are not quite the same as the codification from the service layer so we can't get away with the simplier solution of:

<af:image source="/images/#{bindings.ObjectType}.png" .../> 

So instead, we have to work via an abstraction using a lookup map for the images 

Step 1: Define your image map

We can define the lookup map to work from either as an explicit managed bean, or even more easily as a stand-alone bean definition in the adfc-config.xml file. Conventionally we will store this bean on the application scope so that all users on the system share the same copy:

<managed-bean>
   <managed-bean-name>typeImages</managed-bean-name>
   <managed-bean-class>java.util.HashMap</managed-bean-class>
   <managed-bean-scope>application</managed-bean-scope>
   <map-entries>
     <map-entry>
       <key>TABLE</key>
       <value>/images/table.png</value>
     </map-entry>
     <map-entry>
       <key>VIEW</key>
       <value>/images/view.png</value>
     </map-entry>
     <map-entry>
       <key>FUNCTION</key>
       <value>/images/pls_func.png</value>
     </map-entry>
     ... 
   </map-entries>
</managed-bean>  

Alternatively if the thing we wanted to key off was a simple numerical list then we could have a definition that used a simple array rather than a map; like this:

<managed-bean>
  <managed-bean-name>lifecycleImages</managed-bean-name>
  <managed-bean-class>java.util.ArrayList</managed-bean-class>
  <managed-bean-scope>application</managed-bean-scope>
  <list-entries>
    <value-class>java.lang.String</value-class>
    <value>/images/new.png</value>
    <value>/images/updated.png</value>
    <value>/images/sclosed.png</value>
  </list-entries>
</managed-bean>  

In this latter case index lifecycleImages[0] would map to /images/new.png, lifecycleImages[1] to /images/updated.png etc.

Step 2: using this bean in EL

Now that the list or map has been defined we can use it thus:

<af:image source="#{typeImages[bindings.ObjectType]}"

Where the bindings.ObjectType attribute binding can be expected to turn one of the valid keys in the map (TABLE,VIEW etc.)

 You can see an example of this version of the pattern inside the tile iterator in the home.jspx page in the sample: DRM004 - AutoReduce and Pagination screen

Variations on the Theme

 I'm always thinking about how to optimize performance, and one useful approach when it comes to images is to use the technique of using image sprites. This is where, instead of having lots of discrete images, you have a single image file, a little like a film-strip, containing all of the images. In this case, CSS is used to assign the filmstrip image as a background to a page element and then the CSS background-position is set to select a particular icon on the strip. This has the advantage of only needing a single round trip to grab all the images at once, having a positive effect on your page load time. 

Using this idea we can take the Image Map Pattern approach, but rather than having the map entry value be the name of a discrete image, it can point to the name of the CSS style. This style can be applied in the styleClass attribute of an element to pull the required image from the film-strip. To save you having to create a bunch of extra styles to encode the positions, you could also just hold an offset in the film-strip as the lookup value. 

This latter approach can also be seen in the  DRM004 - AutoReduce and Pagination screen sample, have a look at the managed bean definition for typeImageOffsets. This information is then used to define the images in the list view of the home.jspx page, thus:

<af:spacer styleClass="iconStripBase" inlineStyle="background-position:#{typeImageOffsets[row.ObjectType]}px;"/> 

 Have a look at the sample to see it all in action. 

Thursday Apr 05, 2012

The UIManager Pattern

One of the most common mistakes that I see when reviewing ADF application code, is the sin of storing UI component references, most commonly things like table or tree components in Session or PageFlow scope. The reasons why this is bad are simple; firstly, these UI object references are not serializable so would not survive a session migration between servers and secondly there is no guarantee that the framework will re-use the same component tree from request to request, although in practice it generally does do so.

So there danger here is, that at best you end up with an NPE after you session has migrated, and at worse, you end up pinning old generations of the component tree happily eating up your precious memory. So that's clear, we should never. ever, be storing references to components anywhere other than request scope (or maybe backing bean scope). So double check the scope of those binding attributes that map component references into a managed bean in your applications. 

Why is it Such a Common Mistake? 

At this point I want to examine why there is this urge to hold onto these references anyway? After all, JSF will obligingly populate your backing beans with the fresh and correct reference when needed.  

In most cases, it seems that the rational is down to a lack of distinction within the application between what is data and what is presentation. I think perhaps, a cause of this is the logical separation between business data behind the ADF data binding (#{bindings}) fa├žade and the UI components themselves. Developers tend to think, OK this is my data layer behind the bindings object and everything else is just UI.  Of course that's not the case.  The UI layer itself will have state which is intrinsically linked to the UI presentation rather than the business model, but at the same time should not be tighly bound to a specific instance of any single UI component. So here's the problem.  I think developers try and use the UI components as state-holders for this kind of data, rather than using them to represent that state. An example of this might be something like the selection state of a tabset (panelTabbed), you might be interested in knowing what the currently disclosed tab is. The temptation that leads to the component reference sin is to go and ask the tabset what the selection is.  That of course is fine in context - e.g. a handler within the same request scoped bean that's got the binding to the tabset. However, it leads to problems when you subsequently want the same information outside of the immediate scope.  The simple solution seems to be to chuck that component reference into session scope and then you can simply re-check in the same way, leading of course to this mistake.

Turn it on its Head 

So the correct solution to this is to turn the problem on its head. If you are going to be interested in the value or state of some component outside of the immediate request context then it becomes persistent state (persistent in the sense that it extends beyond the lifespan of a single request). So you need to externalize that state outside of the component and have the component reference and manipulate that state as needed rather than owning it. This is what I call the UIManager pattern. 

Defining the Pattern

The  UIManager pattern really is very simple. The premise is that every application should define a session scoped managed bean, appropriately named UIManger, which is specifically responsible for holding this persistent UI component related state.  The actual makeup of the UIManger class varies depending on a needs of the application and the amount of state that needs to be stored. Generally I'll start off with a Map in which individual flags can be created as required, although you could opt for a more formal set of typed member variables with getters and setters, or indeed a mix. This UIManager class is defined as a session scoped managed bean (#{uiManager}) in the faces-config.xml. 

The pattern is to then inject this instance of the class into any other managed bean (usually request scope) that needs it using a managed property.  So typically you'll have something like this:

  <managed-bean>
    <managed-bean-name>uiManager</managed-bean-name>
    <managed-bean-class>oracle.demo.view.state.UIManager</managed-bean-class>
    <managed-bean-scope>session</managed-bean-scope>
  </managed-bean> 

When is then injected into any backing bean that needs it: 

   <managed-bean>
    <managed-bean-name>mainPageBB</managed-bean-name>
    <managed-bean-class>oracle.demo.view.MainBacking</managed-bean-class>
    <managed-bean-scope>request</managed-bean-scope>
    <managed-property>
      <property-name>uiManager</property-name>
      <property-class>oracle.demo.view.state.UIManager</property-class>
      <value>#{uiManager}</value>
    </managed-property>
  </managed-bean>

In this case the backing bean in question needs a member variable to hold and reference the UIManager:

private UIManager _uiManager; 

Which should be exposed via a getter and setter pair with names that match the managed property name (e.g. setUiManager(UIManager _uiManager), getUiManager()). 

This will then give your code within the backing bean full access to the UI state.

UI components in the page can, of course, directly reference the uiManager bean in their properties, for example, going back to the tab-set example you might have something like this:

<af:paneltabbed>
  <af:showDetailItem text="First"
                     disclosed="#{uiManager.settings['MAIN_TABSET_STATE'].['FIRST']}">
    ...
  </af:showDetailItem>
  <af:showDetailItem text="Second"
                     disclosed="#{uiManager.settings['MAIN_TABSET_STATE'].['SECOND']}">
    ...
  </af:showDetailItem>
  ...
</af:panelTabbed>

Where in this case the settings member within the UI Manger is a Map which contains a Map of Booleans for each tab under the MAIN_TABSET_STATE key. (Just an example you could choose to store just an identifier for the selected tab or whatever, how you choose to store the state within UI Manger is up to you.)

Get into the Habit

So we can see that the UIManager pattern is not great strain to implement for an application and can even be retrofitted to an existing application with ease. The point is, however, that you should always take this approach rather than committing the sin of persistent component references which will bite you in the future or shotgun scattered UI flags on the session which are hard to maintain.  If you take the approach of always accessing all UI state via the uiManager, or perhaps a pageScope focused variant of it, you'll find your applications much easier to understand and maintain. Do it today!

More Information

Another interesting article relating to this topic has been written by Steven Davelaar  subsequent to the original publication of this post. This article is well worth checking out more more information on this area and some best practice around component references.

Wednesday Mar 07, 2012

Optimizing Secured Hudson performance on Tomcat

There was some debate in the Hudson Community meeting this week on various areas of performance that need to be addressed for enterprise grade installations such as that used at the Eclipse foundation. Least of these issues was a problem that we'd observed both on ci.hudson-ci.org server and other, and this was the matter of image and other resource caching, or rather, lack thereof.  Hudson itself has a filter which sets the expire header on all of these resources to a year in the future, so the browser should be retrieving from cache rather than hitting the server again. However, this simply was not happening.

An investigation of the traffic via Chrome's developer tools shows the cause:

screenshot from chrome

 You can see that the Expires directive in the response is 2013, however, the Cache-Control: no-cache is overriding this.

It turns out that this is down to Tomcat behaviour when running a secured web app. It sets the no-cache directive automatically, irrespective of the type of resource it is serving. Fortunately there is a way to prevent this behaviour  using a simple configuration in the Tomcat server.xml, simply add the following Valve definition into the <Context> being used for Hudson:

<Valve className="org.apache.catalina.authenticator.NonLoginAuthenticator"
       disableProxyCaching="false" />

Once Tomcat is restarted, your resources will be cache-able again. For more complete information about Hudson configuration behind Tomcat check out the Tomcat page in the Hudson Wiki.

Wednesday Dec 21, 2011

ADF Performance Presentation from UKOUG

I've given this presentation a few times now at various events around the world, most recently at the UK Oracle Users Group. It's an ever evolving topic area so I'm sure the paper will change over time, but here's the version for now:

This is very much based on our experience of tuning real ADF Enterprise Applications, for both internal and external customers.  Hopefully there are a few useful nuggets of information in there for you.

Thursday Nov 24, 2011

JSP Precompilation for ADF Applications

A question that comes up from time to time, particularly in relation to build automation, is how to best pre-compile the .jspx and .jsff files in an ADF application. Thus ensuring that the app is ready to run as soon as it's installed into WebLogic. In the normal run of things, the first poor soul to hit a page pays the price and has to wait a little whilst the JSP is compiled into a servlet. Everyone else subsequently gets a free lunch. So it's a reasonable thing to want to do...

Let Me List the Ways

So forth to Google (other search engines are available)... which lead me to a fairly old article on WLDJ - Removing Performance Bottlenecks Through JSP Precompilation. Technololgy wise, it's somewhat out of date, but the one good point that it made is that it's really not very useful to try and use the precompile option in the weblogic.xml file. That's a really good observation - particularly if you're trying to integrate a pre-compile step into a Hudson Continuous Integration process. That same article mentioned an alternative approach for programmatic pre-compilation using weblogic.jspc. This seemed like a much more useful approach for a CI environment. However, weblogic.jspc is now obsoleted by weblogic.appc so we'll use that instead.  Thanks to Steve for the pointer there.

And So To APPC

APPC has documentation - always a great place to start, and supports usage both from Ant via the wlappc task and from the command line using the weblogic.appc command. In my testing I took the latter approach.

Usage, as the documentation will show you, is superficially pretty simple.  The nice thing here, is that you can pass an existing EAR file (generated of course using OJDeploy) and that EAR will be updated in place with the freshly compiled servlet classes created from the JSPs. Appc takes care of all the unpacking, compiling and re-packing of the EAR for you. Neat. 

So we're done right...? Not quite.

The Devil is in the Detail

 OK so I'm being overly dramatic but it's not all plain sailing, so here's a short guide to using weblogic.appc to compile a simple ADF application without pain. 

Information You'll Need

The following is based on the assumption that you have a stand-alone WLS install with the Application Development  Runtime installed and a suitable ADF enabled domain created. This could of course all be run off of a JDeveloper install as well

1. Your Weblogic home directory. Everything you need is relative to this so make a note.  In my case it's c:\builds\wls_ps4.

2. Next deploy your EAR as normal and have a peek inside it using your favourite zip management tool. First of all look at the weblogic-application.xml inside the EAR /META-INF directory. Have a look for any library references. Something like this:

<library-ref>
   <library-name>adf.oracle.domain</library-name>
</library-ref> 

 Make a note of the library ref (adf.oracle.domain in this case) , you'll need that in a second.

3. Next open the nested WAR file within the EAR and then have a peek inside the weblogic.xml file in the /WEB-INF directory. Again  make a note of the library references.

4. Now start the WebLogic as per normal and run the WebLogic console app (e.g. http://localhost:7001/console). In the Domain Structure navigator, select Deployments.

5. For each of the libraries you noted down drill into the library definition and make a note of the .war, .ear or .jar that defines the library. For example, in my case adf.oracle.domain maps to "C:\ builds\ WLS_PS4\ oracle_common\ modules\ oracle. adf. model_11. 1. 1\ adf. oracle. domain. ear". Note the extra spaces that are salted throughout this string as it is displayed in the console - just to make it annoying, you'll have to strip these out.

6. Finally you'll need the location of the adfsharebean.jar. We need to pass this on the classpath for APPC so that the ADFConfigLifeCycleCallBack listener can be found. In a more complex app of your own you may need additional classpath entries as well. 

Now we're ready to go, and it's a simple matter of applying the information we have gathered into the relevant command line arguments for the utility

A Simple CMD File to Run APPC 

Here's the stub .cmd file I'm using on Windows to run this.

@echo off
REM Stub weblogic.appc Runner
setlocal

set WLS_HOME=C:\builds\WLS_PS4
set ADF_LIB_ROOT=%WLS_HOME%\oracle_common\modules
set COMMON_LIB_ROOT=%WLS_HOME%\wlserver_10.3\common\deployable-libraries
set ADF_WEBAPP=%ADF_LIB_ROOT%\oracle.adf.view_11.1.1\adf.oracle.domain.webapp.war
set ADF_DOMAIN=%ADF_LIB_ROOT%\oracle.adf.model_11.1.1\adf.oracle.domain.ear
set JSTL=%COMMON_LIB_ROOT%\jstl-1.2.war
set JSF=%COMMON_LIB_ROOT%\jsf-1.2.war
set ADF_SHARE=%ADF_LIB_ROOT%\oracle.adf.share_11.1.1\adfsharembean.jar

REM Set up the WebLogic Environment so appc can be found
call %WLS_HOME%\wlserver_10.3\server\bin\setWLSEnv.cmd
CLS

REM Now compile away!
java weblogic.appc -verbose -library %ADF_WEBAPP%,%ADF_DOMAIN%,%JSTL%,%JSF% -classpath %ADF_SHARE% %1

endlocal

Running the above on a target ADF .ear  file will zip through and create all of the relevant compiled classes inside your nested .war file in the \WEB-INF\classes\jsp_servlet\ directory (but don't take my word for it, run it and take a look!)

And So...

In the immortal words of  the Pet Shop Boys, Was It Worth It? Well, here's where you'll have to do your own testing. In  my case here, with a simple ADF application, pre-compilation shaved an non-scientific "3 Elephants" off of the initial page load time for the first access of each page. That's a pretty significant payback for such a simple step to add into your CI process, so why not give it a go.

About

Hawaii, Yes! Duncan has been around Oracle technology way too long but occasionally has interesting things to say. He works in the Development Tools Division at Oracle, but you guessed that right? In his spare time he contributes to the Hudson CI Server Project at Eclipse
Follow DuncanMills on Twitter

Note that comments on this blog are moderated so (1) There may be a delay before it gets published (2) I reserve the right to ignore silly questions and comment spam is not tolerated - it gets deleted so don't even bother, we all have better things to do with our lives.
However, don't be put off, I want to hear what you have to say!

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today