Friday Apr 19, 2013

Lions and Tigers and RangeSize, Oh My!

One of the most common mistakes I see made in ADF Business Components based applications is a failure to tune the View Objects, and specifically to tune the in Batches of parameter in the VO tuning section. This setting defaults to 1, which does not always meet the needs of the consuming UI or service interface and should generally be changed.  This is a topic that I and others have covered before.

Now certainly the batch fetch size is a crucial tuning parameter as it controls how many times the framework has to hit the database to get the rows needed to service a particular function.  However, in this posting I wanted to turn my attention to another factor which can also have a significant effect - the iterator RangeSize.

The background to this was some recent detective work on an application where the time taken to display one particular screen was suspiciously long. 
The page in question had a tabular display of data, but an inspection of the VO tuning parameters showed that a reasonable Batch size of 51 was being used. What's more the As Needed switch rather than the All at Once option in the VO tuning was being used. So the developer had done totally the right things there.
Running a SQL Trace on the page revealed an interesting thing though.  Because the batch size was pretty high we'd expect that the framework would have to only do one or at most two fetches from the cursor to satisfy the needs of that table.  However The TKProf output showed that in fact over 150 fetches took place retrieving over 8000 rows!

My thought processes in diagnosing this were to look in the following places:
  1. Are there alternative VO Instances defined on the AM where the tuning parameters are different (e.g. ALL_ROWS was specified)? We know the base definition is OK but it could be overridden.  
  2. Any programmatic calls to change the Batch Size or fetch node in the VO?
  3. Any programatic calls to last() on the rowset or iterations through the rowset? 
  4. Check for a RangeSize of -1 on the iterator definition in the pageDef files.

All of these drew a blank.  The last one in particular felt like the problem but a search for the value of -1 in the pageDefs of the UI project only turned up legitimate usages of the -1 value. 

Hold on I don't Understand This RangeSize?

Maybe I should take a quick step back and explain the iterator RangeSize.  So, as we've seen, the tuning options in the view Object will control how often the VO has to go back to the database to get a specific number of rows. The iterator rangeSize is defined in the pageDef file for a particular page, fragment or method activity and it defines how many rows the UI layer should ask the service layer (in this case the VO) for. 
Here's a typical definition that you'll see in the pageDef:
<iterator Binds="EmployeesView1" 

You'll see that the rangeSize here is set to 25 which just happens to be the value that is defaulted in when you drag and drop a binding into the page.  However, it turns out that 25 is not the default value, something which has a bearing later in this investigation as we'll see.
So in this default case when the iterator is asked for data, it in turn will ask the VO for 25 rows, and if the VO does not already have that many rows in the cache it will have to go back to the database as many times, as determined by the batch-size, as it needs to get enough data.

Back to the Problem page 

As it happens the pageDef for the table displaying the problem VO was indeed the defacto default of 25, so, sad to say, it was not the obvious suspect at fault, more investigation was needed.

At this stage the investigation splits into a couple of parallel efforts, manual code inspection, and tracing using the ADF Logging capability to try and work out what interactions were happening with the problem VO.

Welcome to the Internet, Please Tread with Care

What can we trust? Well in the world of ADF Blogs in the wild there are some great bloggers, but that does not mean that you can just copy without thinking.  It turned out that one of the problems with this application was to fall foul of copy-without-thinking syndrome.
The code in question seems innocent enough, it's published out there on the internet as a way of refreshing an iterator:

//Please, Please Don't do this! (My comment)
DCIteratorBinding iterBind= (DCIteratorBinding)dc.get("<your iterator name>");  

Two facts to discuss in relation to this code snippet:
  1. Read the JavaDoc for DCIteratorBinding - the refresh() method is very clearly marked as Internal only, applications should not use. That's what we call "a hint" in the trade, you can choose to ignore it but don't come crying...
  2. Look at that parameter being passed to the refresh method DCIteratorBinding.RANGESIZE_UNLIMITED - can you guess what that does? Yes it does the same as setting the RangeSize to -1 and will cause the VO to be asked for all of the data for that query. You can see how bad that could be if the VO has the potential to return a lot of rows.
So something to put right already. 

But Wait, There's more!

Although the call to refresh was a great catch and the application will be better without it, it turned out not to be the cause - darn. 
However, the parellel effort to run some more tracing found the smoking gun.  The ADF Log trace showed a double execute on the iterator for the VO in question, or to be more precise, executes on two different iterators bound to the same VO from different regions on the page. 
A useful diagnostic here was then to set a breakpoint in the setRangeSize() of oracle.adf.model.binding.DCIteratorBinding.  Doing this we could see that the first iterator execution was actually responsible for setting the RangeSize to -1 and the second to the value we where expecting for that table based on the pageDef. 
All credit to the development team I was working with here who ferreted out the actual problem, it was finally down to one of omission. 

Recall I made the statement earlier about 25 being the defacto default for the RangeSize? Very true, when you create a binding, that's what the IDE puts in for you.  But what's the actual default? Well that turns out to be -1.  So if you omit the RangeSize from the iterator definition by intent, or mistake, you're going to have a potential side effect you may not have expected!  That was exactly the problem in this case  - no RangeSize. 

Specifically the problem was caused by a method binding in one of the bounded task flows in the page. The pageDef for this method activity included and iterator defined for the problem VO but without a RangeSize defined.  

Lessons to Learn

  1. For each peice of UI that is bound to data understand how the user is going to use that screen - will they view just one record, will they scroll through several or a lot, will they only scroll downwards and process one record at a time and so on. You can't effectivly tune a page until you understand how it will be used.
  2. Always tune your VO definitions or instances to the requirements of each endpoint.  If this means having multiple VO definitions or multiple VO instances with different tuning parameters then do it.
  3. If you have consciously set your Batch size in the VO tuning parameters to a small number because that's all you see on the page at once then also tune the pageDef iterator RangeSize. Otherwise that defacto 25 RangeSize could generate a bunch of unnecessary calls back to the database. Think about it, if you have a single record form and you've not tuned the VO Batch size and you've not tuned the iterator RangeSize, you could be doing 25 roundtrips to the database just to see one record.  Great way to please your DBA!
  4. Use a RangeSize of -1 with caution.  It has its place for iterators that serve short lists used for lookups and menus but anything else is an exception.
  5. Don't blindly copy everything you see in the blog-o-sphere. If you don't recognize an API call then look it up.  If something says it's for internal use only, then guess what, don't use it.
  6. Never ever, define an iterator in your pageDef without an explicit rangeSize. If you need to see all rows, say so with the -1 value, otherwise use a positive integer.

We all go home older and wiser...

Monday Feb 18, 2013

MySQL & ADF Business Components - Enum types

A quick guide to effectively mapping and representing MySQL enumeration types through ADF Business Components [Read More]

Friday Jan 18, 2013

Refresh Problems using Adaptive Bindings

In a previous article (Towards Ultra-Reusability for ADF - Adaptive Bindings)  I discussed how ADF Data Binding can be a lot more flexible that you would think due to the neat trick of being able to use expression language within the PageDef file to create bind sources that  can be switched at runtime.

As it happens my good buddy Frank Nimphius picked up the technique to use on a particular  project and hit a slight problem. If you change the results of the EL used in the binding - for example, you switch the base VO from Departments to Employees, things don't get refreshed correctly.  In fact what you see is that any attribute names that happen to match between the old and the new source will be correctly populated with the new data but the actual list of attributes will not be refreshed. The effect is that if you were using one of these bindings to populate a table, the "shape" of the table, in terms of its columns, would not change. 

No worries though, given that Frank is a clever chap he worked out the correct way to address this which is to simply call clearForRecreate() on the iterator binding.

 BindingContext bctx = BindingContext.getCurrent();
 BindingContainer bindings = bctx.getCurrentBindingsEntry();
 DCIteratorBinding iter = (DCIteratorBinding)

Thanks Frank! 

Saturday Nov 17, 2012

Towards Ultra-Reusability for ADF - Adaptive Bindings

The task flow mechanism embodies one of the key value propositions of the ADF Framework, it's primary contribution being the componentization of your applications and implicitly the introduction of a re-use culture, particularly in large applications.

However, what if we could do more? How could we make task flows even more re-usable than they are today? Well one great technique is to take advantage of a feature that is already present in the framework, a feature which I will call, for want of a better name, "adaptive bindings".

What's an adaptive binding? well consider a simple use case.  I have several screens within my application which display tabular data which are all essentially identical, the only difference is that they happen to be based on different data collections (View Objects, Bean collections, whatever) , and have a different set of columns. Apart from that, however, they happen to be identical; same toolbar, same key functions and so on. So wouldn't it be nice if I could have a single parametrized task flow to represent that type of UI and reuse it?

Hold on you say, great idea, however, to do that we'd run into problems. Each different collection that I want to display needs different entries in the pageDef file and:

  1. I want to continue to use the ADF Bindings mechanism rather than dropping back to passing the whole collection into the taskflow  
  2. If I do use bindings, there is no way I want to have to declare iterators and tree bindings for every possible collection that I might want the flow to handle

 Ah, joy! I reply, no need to panic, you can just use adaptive bindings.

Defining an Adaptive Binding 

It's easiest to explain with a simple before and after use case.  Here's a basic pageDef definition for our familiar Departments table. 

  <iterator Binds="DepartmentsView1" 
  <tree IterBinding="DepartmentsView1Iterator" id="DepartmentsView1">
    <nodeDefinition DefName="oracle.demo.model.vo.DepartmentsView" Name="DepartmentsView10">
        <Item Value="DepartmentId"/>
        <Item Value="DepartmentName"/>
        <Item Value="ManagerId"/>
        <Item Value="LocationId"/>

Here's the adaptive version:

  <iterator Binds="${pageFlowScope.voName}" 
  <tree IterBinding="TableSourceIterator" id="GenericView">
      <nodeDefinition Name="GenericViewNode"/>

You'll notice three changes here.  

  1. Most importantly, you'll see that the hard-coded View Object name  that formally populated the iterator Binds attribute is gone and has been replaced by an expression (${pageFlowScope.voName}). This of course, is key, you can see that we can pass a parameter to the task flow, telling it exactly what VO to instantiate to populate this table!
  2. I've changed the IDs of the iterator and the tree binding, simply to reflect that they are now re-usable
  3. The tree binding itself has simplified and the node definition is now empty.  Now what this effectively means is that the #{node} map exposed through the tree binding will expose every attribute of the underlying iterator's collection - neat! (kudos to Eugene Fedorenko at this point who reminded me that this was even possible in his excellent "deep dive" session at OpenWorld  this year)

Using the adaptive binding in the UI

Now we have a parametrized  binding we have to make changes in the UI as well, first of all to reflect the new ID that we've assigned to the binding (of course) but also to change the column list from being a fixed known list to being a generic metadata driven set:

<af:table value="#{bindings.GenericView.collectionModel}"
          emptyText="#{bindings.GenericView.viewable ? 'No data to display.' : 'Access Denied.'}"
          var="row" rowBandingInterval="0" 
          rowSelection="single" id="t1">
  <af:forEach items="#{bindings.GenericView.attributeDefs}" var="def">
    <af:column headerText="#{bindings.GenericView.labels[]}" sortable="true"
               sortProperty="#{}" id="c1">
      <af:outputText value="#{row.bindings[].inputValue}" id="ot1"/>

Of course you are not constrained to a simple read only table here.  It's a normal tree binding and iterator that you are using behind the scenes so you can do all the usual things, but you can see the value of using ADFBC as the back end model as you have the rich pantheon of UI hints to use to derive things like labels (and validators and converters...) 

One Final Twist

 To finish on a high note I wanted to point out that you can take this even further and achieve the ultra-reusability I promised. Here's the new version of the pageDef iterator, see if you can notice the subtle change?

<iterator Binds="{pageFlowScope.voName}" 

Yes, as well as parametrizing the collection (VO) name, we can also parametrize the name of the data control. So your task flow can graduate from being re-usable within an application to being truly generic. So if you have some really common patterns within your app you can wrap them up and reuse then across multiple developments without having to dictate data control names, or connection names. This also demonstrates the importance of interacting with data only via the binding layer APIs. If you keep any code in the task flow generic in that way you can deal with data from multiple types of data controls, not just one flavour. Enjoy!


Read this post as well on overcoming possible refresh problems when changing the source on a single page. 

Further update

Check out this article from Luc Bors on using similar ideas with Query Components / View Criteria.  

Thursday Oct 18, 2012

ADF Logging In Deployed Apps

Harking back to my series on using the ADF logger and the related  ADF Insider Video, I've had a couple of queries this week about using the logger from Enterprise Manager (EM). I've alluded in those previous materials to how EM can be used but it's evident that folks need a little help.  So in this article, I'll quickly look at how you can switch logging on from the EM console for an application and how you can view the output. 

Before we start I'm assuming that you have EM up and running, in my case I have a small test install of Fusion Middleware Patchset 5 with an ADF application deployed to a managed server.

Step 1 - Select your Application

In the EM navigator select the app you're interested in:

At this point you can actually bring up the context ( right mouse click) menu to jump to the logging, but let's do it another way. 

Step 2 - Open the Application Deployment Menu

At the top of the screen, underneath the application name, you'll find a drop down menu which will take you to the options to view log messages and configure logging, thus:

Step 3 - Set your Logging Levels 

Just like the log configuration within JDeveloper, we can set up transient or permanent (not recommended!) loggers here.

In this case I've filtered the class list down to just oracle.demo, and set the log level to config. You can now go away and do stuff in the app to generate log entries.

Step 4 - View the Output 

Again from the Application Deployment menu we can jump to the log viewer screen and, as I have here, start to filter down the logging output to the stuff you're interested in. 

In this case I've filtered by module name. You'll notice here that you can again look at related log messages.

Importantly, you'll also see the name of the log file that holds this message, so it you'd rather analyse the log in more detail offline, through the ODL log analyser in JDeveloper, then you can see which log to download.

Friday Oct 05, 2012

ADF - Now with Robots!

I mentioned this briefly in a tweet the other day, just before the full rush of OOW really kicked off, so I though it was worth re-visiting. Check out this video, and then read on:

So why so interesting? Well - you probably guessed from the title, ADF is involved. Indeed this is as about as far from the traditional ADF data entry application as you can get. Instead of a database at the back-end there's basically a robot. That's right, this remarkable tape drive is controlled through an ADF using all your usual friends of ADF Faces, Controller and Binding (but no ADFBC for obvious reasons). ADF is used both on the touch screen you see on the front of the device in the video, and also for the remote management console which provides a visual representation of the slots and drives. The latter uses ADF's Active Data Framework to provide a real-time view of what's going on the rack.

SL150 GUI Screen Shot.

What's even more interesting (for the techno-geeks) is the fact that all of this is running out of flash storage on a ridiculously small form factor with tiny processor - I probably shouldn't reveal the actual specs but take my word for it, don't complain about the capabilities of your laptop ever again!

This is a project that I've been personally involved in and I'm pumped to see such a good result and,  I have to say, those hardware guys are great to work with (and have way better toys on their desks than we do).

More info in the SL150 (should you feel the urge to own one) is here

Tuesday Sep 04, 2012

forEach and Facelets - a bugfarm just waiting for harvest

An issue that I've encountered before and saw again today seems worthy of a little write-up. It's all to do with a subtle yet highly important difference in behaviour between JSF 2 running with JSP and running on Facelets (.jsf pages). The incident I saw today can be seen as a report on the ADF EMG bugzilla (Issue 53) and in a blog posting by Ulrich Gerkmann-Bartels who reported the issue to the EMG. Ulrich's issue nicely shows how tricky this particular gochya can be. On the surface, the problem is squarely the fault of MDS but underneath MDS is, in fact, innocent.

To summarize the problem in a simpler testcase than Ulrich's example, here's a simple fragment of code:

<af:forEach var="item" items="#{itemList.items}" varStatus="vs">
  <af:commandLink id="cl1" text="#{item.label}" action="#{item.doAction}" 

Looks innocent enough right? We see a bunch of links printed out, great.

The issue here though is the id attribute. Logically you can kind of see the problem. The forEach loop is creating (presumably) multiple instances of the commandLink, but only one id is specified - cl1. We know that IDs have to be unique within a JSF component tree, so that must be a bad thing?  The problem is that JSF under JSP implements some hacks when the component tree is generated to transparently fix this problem for you. Behind the scenes it ensures that each instance really does have a unique id. Really nice of it to do so, thank you very much.

However, (you could see this coming), the same is not true when running with Facelets  (this is under 11.1.2.n)  in that case, what you put for the id is what you get, and JSF does not mess around in the background for you. So you end up with a component tree that contains duplicate ids which are only created at runtime.  So subtle chaos can ensue.  The symptoms are wide and varied, from something pretty obscure such as the combination Ulrich uncovered, to something as frustrating as your ActionListener just not being triggered. And yes I've wasted hours on just such an issue. 

The Solution 

Once you're aware of this one it's really simple to fix it, there are two options:

  1. Remove the id attribute on components that will cause some kind of submission within the forEach loop altogether and let JSF do the right thing in generating them. Then you'll be assured of uniqueness.
  2. Use the var attribute of the loop to generate a unique id for each child instance.  for example in the above case: <af:commandLink id="cl1_${vs.index}" ... />.

 So one to watch out for in your upgrades to JSF 2 and one perhaps, for your coding standards today to prepare you for.

For completeness, here's the reference to the underlying JSF issue that's at the heart of this: JAVASERVERFACES-1527

Monday Jul 09, 2012

ADF and EBS Applications

A blog entry that may be of interest to those of using building ADF apps that, in some way, need to integrate with Oracle E-Business Suite. Head over to Steven Chan's Applications Technology Blog: Building Extensions Using E-Business Suite SDK for Java

Tuesday Jul 03, 2012

New Sample Demonstrating the Traversing of Tree Bindings

A technique that I seem to use a fair amount, particularly in the construction of dynamic UIs is the use of a ADF Tree Binding to encode a multi-level master-detail relationship which is then expressed in the UI in some kind of looping form – usually a series of nested af:iterators, rather than the conventional tree or treetable. This technique exploits two features of the treebinding. First the fact that an treebinding can return both a collectionModel as well as a treeModel, this collectionModel can be used directly by an iterator. Secondly that the “rows” returned by the collectionModel themselves contain an attribute called .children. This attribute in turn gives access to a collection of all the children of that node which can also be iterated over.

Putting this together you can represent the data encoded into a tree binding in all sorts of ways.

As an example I’ve put together a very simple sample based on the HT schema and uploaded it to the ADF Sample project. It produces this UI:

Example output from this technique

The important code is shown here for a Region -> Country -> Location Hierachy:

<af:iterator id="i1" value="#{bindings.AllRegions.collectionModel}" var="rgn">
  <af:showDetailHeader text="#{rgn.RegionName}" disclosed="true" id="sdh1">
    <af:iterator id="i2" value="#{rgn.children}" var="cnty">
      <af:showDetailHeader text="#{cnty.CountryName}" disclosed="true" id="sdh2">
        <af:iterator id="i3" value="#{cnty.children}" var="loc">
          <af:panelList id="pl1">
            <af:outputText value="#{loc.City}" id="ot3"/>

You can download the entire sample from here:

Thursday Apr 05, 2012

The UIManager Pattern

One of the most common mistakes that I see when reviewing ADF application code, is the sin of storing UI component references, most commonly things like table or tree components in Session or PageFlow scope. The reasons why this is bad are simple; firstly, these UI object references are not serializable so would not survive a session migration between servers and secondly there is no guarantee that the framework will re-use the same component tree from request to request, although in practice it generally does do so.

So there danger here is, that at best you end up with an NPE after you session has migrated, and at worse, you end up pinning old generations of the component tree happily eating up your precious memory. So that's clear, we should never. ever, be storing references to components anywhere other than request scope (or maybe backing bean scope). So double check the scope of those binding attributes that map component references into a managed bean in your applications. 

Why is it Such a Common Mistake? 

At this point I want to examine why there is this urge to hold onto these references anyway? After all, JSF will obligingly populate your backing beans with the fresh and correct reference when needed.  

In most cases, it seems that the rational is down to a lack of distinction within the application between what is data and what is presentation. I think perhaps, a cause of this is the logical separation between business data behind the ADF data binding (#{bindings}) façade and the UI components themselves. Developers tend to think, OK this is my data layer behind the bindings object and everything else is just UI.  Of course that's not the case.  The UI layer itself will have state which is intrinsically linked to the UI presentation rather than the business model, but at the same time should not be tighly bound to a specific instance of any single UI component. So here's the problem.  I think developers try and use the UI components as state-holders for this kind of data, rather than using them to represent that state. An example of this might be something like the selection state of a tabset (panelTabbed), you might be interested in knowing what the currently disclosed tab is. The temptation that leads to the component reference sin is to go and ask the tabset what the selection is.  That of course is fine in context - e.g. a handler within the same request scoped bean that's got the binding to the tabset. However, it leads to problems when you subsequently want the same information outside of the immediate scope.  The simple solution seems to be to chuck that component reference into session scope and then you can simply re-check in the same way, leading of course to this mistake.

Turn it on its Head 

So the correct solution to this is to turn the problem on its head. If you are going to be interested in the value or state of some component outside of the immediate request context then it becomes persistent state (persistent in the sense that it extends beyond the lifespan of a single request). So you need to externalize that state outside of the component and have the component reference and manipulate that state as needed rather than owning it. This is what I call the UIManager pattern. 

Defining the Pattern

The  UIManager pattern really is very simple. The premise is that every application should define a session scoped managed bean, appropriately named UIManger, which is specifically responsible for holding this persistent UI component related state.  The actual makeup of the UIManger class varies depending on a needs of the application and the amount of state that needs to be stored. Generally I'll start off with a Map in which individual flags can be created as required, although you could opt for a more formal set of typed member variables with getters and setters, or indeed a mix. This UIManager class is defined as a session scoped managed bean (#{uiManager}) in the faces-config.xml. 

The pattern is to then inject this instance of the class into any other managed bean (usually request scope) that needs it using a managed property.  So typically you'll have something like this:


When is then injected into any backing bean that needs it: 


In this case the backing bean in question needs a member variable to hold and reference the UIManager:

private UIManager _uiManager; 

Which should be exposed via a getter and setter pair with names that match the managed property name (e.g. setUiManager(UIManager _uiManager), getUiManager()). 

This will then give your code within the backing bean full access to the UI state.

UI components in the page can, of course, directly reference the uiManager bean in their properties, for example, going back to the tab-set example you might have something like this:

  <af:showDetailItem text="First"
  <af:showDetailItem text="Second"

Where in this case the settings member within the UI Manger is a Map which contains a Map of Booleans for each tab under the MAIN_TABSET_STATE key. (Just an example you could choose to store just an identifier for the selected tab or whatever, how you choose to store the state within UI Manger is up to you.)

Get into the Habit

So we can see that the UIManager pattern is not great strain to implement for an application and can even be retrofitted to an existing application with ease. The point is, however, that you should always take this approach rather than committing the sin of persistent component references which will bite you in the future or shotgun scattered UI flags on the session which are hard to maintain.  If you take the approach of always accessing all UI state via the uiManager, or perhaps a pageScope focused variant of it, you'll find your applications much easier to understand and maintain. Do it today!

More Information

Another interesting article relating to this topic has been written by Steven Davelaar  subsequent to the original publication of this post. This article is well worth checking out more more information on this area and some best practice around component references.


Hawaii, Yes! Duncan has been around Oracle technology way too long but occasionally has interesting things to say. He works in the Development Tools Division at Oracle, but you guessed that right? In his spare time he contributes to the Hudson CI Server Project at Eclipse
Follow DuncanMills on Twitter

Note that comments on this blog are moderated so (1) There may be a delay before it gets published (2) I reserve the right to ignore silly questions and comment spam is not tolerated - it gets deleted so don't even bother, we all have better things to do with our lives.
However, don't be put off, I want to hear what you have to say!


« April 2014