Oracle BI Publisher Consulting

Hi all,

If you're subscribing this blog through Email subscription please go to the new blog site and re-register your subscription by using the Subscription box that is located at right hand side on the new page. Somehow the migration of the blog site doesn't work for the existing Email subscription.... Sorry for inconvenience...

New BI Publisher Consulting Blog Site
or,
http://bipconsulting.blogspot.com/

We have decided to move this BI Publisher Consulting blog to a new blog hosting service. Here is a new address, so please renew your bookmark or blog reader's address.

Oracle BI Publisher Consulting

or

http://bipconsulting.blogspot.com/

Sorry for inconvenience... But the new blog hosting service is much easier to maintain so we'll have more contents coming in and more quickly. Please stay tuned!

When you start developing RTF Templates and try to do something a bit advanced most likely you start seeing these words, XSL, XSLT, XPATH, XSL-FO…. And they might have scared or confused you. I’ve met many people use those terms without a correct understanding and everybody uses them in different ways so even I get sometimes confused when I talk with them. ;-) I think the part of the reason is coming from its own history.

XSL stands for ‘Extensible Stylesheet Language’ and it was designed to describe how to transform and format XML files. But it split into different specifications (or languages) as listed below. So this XSL itself actually is really anything today, it rather be a family name if you will, which contains all the three related languages below.

Here is a list of the three languages that comprise the XSL.

XPATH – a language for navigating in XML documents
XSLT – a language for transforming XML documents
XSL-FO – a language for formatting XML documents

Let’s take at look at each language in more detail and why we need to know them for BI Publisher reports development.

Start from XPath…


XPath is a language for finding information in an XML document. XPath is used to navigate through elements and attributes in an XML document. XPath uses path expressions to navigate in XML documents. It is very similar to what we call ‘Path’ on Unix file system. Basically it acts as a navigation in XML files. Let’s say there is a parent node called ‘Department’ and it has a child node called ‘Employee’. Now you’re processing Employee node and want to get some data from the parent node, Department. This is where the XPath comes in. And it is in fact very simple to do this. You can type something like ‘../Department/BUDGET’. Yes, that’s it and it’s very similar to the path we use at the file system, right?

Path Expression

What makes XPath different from the path is its Path Expression. The Path Expression is very powerful and makes it much easier to access to any vaues in the XML files.
For example you can use ‘//’ (double slashes) to indicate that you want to get any Element and it doesn’t matter if what level the element is.

Here is a list of commonly used Path Expressions.

/

Selects from the root node

Example:
/Department (Selects the root element Department)
Department/Employee (Selects all Employee elements that are children of Department)

//

Selects nodes in the document from the current node that match the selection no matter where they are

Example:
//Department (Selects all Department elements no matter where they are in the XML)
Department//Employee (Selects all Employee elements that are descendant of the Department element, no matter where they are under the Department element)

.

Selects the current node

..

Selects the parent of the current node

@

Selects attributes

Example:
//@type (Selects all attributes that are named type)

Predicates

Also, there are some advanced expressions called ‘Predicates’. Predicates are something you might have seen in your template They are presented with square brackets. They are used to find a specific node and can have specific conditions to specify the node. For example, ‘/Department[1]’ will return the first Department node while ‘/Department[2]’ will return the second Department node. You can also have a condition in the predicates to pick a certain set of nodes only when the condition matches. For example, if you specify ‘/Department[Salary>5000]’ then it will return only the Department nodes that contains Salary element whose values are greater than 5000.

You can also specify relative position. The above example of ‘/Department[1]’ will always return the first Department node in the XML file. But if you have many groups or you’re grouping by a certain value (e.g. Department name) and you might want to pick the first node in the each group. In this case you can use ‘Department[first()]’ or ‘Department[position()=1].

Operators (Functions)

Also, XPath has its own Operators (or functions) that you can use to process or calculate your data in the XML file. For example there is a ‘substring’ function, which you can use to get a part of the data you want from a specified Element values. There are many other useful functions and all of the standard XPath functions can be used in the BI Publisher’s RTF Template. Here is a set of XPATH functions that are useful and we use in many cases with the RTF Template.

List of XPATH functions

- substring()
- substring-before('12/10','/')
- replace("Bella Italia", "l", "")
- upper-case() /lower-case()
- contains()
- distinct-values()
- false()
- sum()

Why this is for BI Publisher?

Now you have gone though the XPath basic and wondering ‘why do I need to know this?’ Here is list of example use cases where we think it is critical and very useful if you understand the XPath appropriately.

• With for-each-group you need to specify a parent node and an element node where you want to group by with XPath appropriately
• When you want to access to a parent node’s element values when you are processing tis child node.
• For IF condition you might want to process data to build some valid conditions. For example if you want to get Employee name ‘Smith’ regardless whether it’s in upper case, lower case or the combination, you can use ‘upper-case()’ so that every type of ‘Smith’ will be matched.
• Inside Chart definition you need to specify all the Element names appropriately with XPath. Also you can use the XPath functions to have conditions or process data inside the Chart.

These are just a few examples to list, but there are many other cases you can take advantage of the XPath and make your RTF Template development much easier and more flexible and powerful.

Also, note that using the XPath without an appropriate understanding might cause performance and resource allocation problems. Due to its flexibility you can achieve what you want to do by using many different ways with XPath. However, you might end up typing XPath codes that cause very resource intensive or unnecessary processing. The key is to have a right understanding of XPath and write the code in the most optimized way from maintenance and performance perspective.

At the next session, we’ll cover XSLT & XSL-FO. Stay tuned!

Rules to Remember

We have covered the basic knowledge of XML. Now there are some standard rules you need to follow when you work with XML files. Any validation of the rules will throw you an error by any standard XML processor including the one used by BI Publisher. Good news is, XML has only a few rules and all the rules actually make sense and most importantly they are easy enough to follow.

So here is a list of main rules to remember for you at least.

- All XML Elements must have a closing tag
- Element and Attribute names are case sensitive
- Element must be properly nested
- Documents must have a root element
- Attribute Values must be quoted
- White Space value is preserved
- Element/Attribute Values can contain letters, numbers, and other characters
- Element Name Must not start with a number or punctuation character
- Element Name Must not start with the letters ‘xml’ (or XML)
- Element Name cannot contain spaces

I’ve seen many users at our clients have encountered errors when they developed or run BI Publisher reports (RTF template or Data template) because of the non-compliance with the above rules. As you know currently we still need to develop the Data Template files manually and many of us uses text editors to edit the XML format files. And we make some typo, forgot single/double quotes, mistakenly put a space in the element name, or forgot to end the node with closing element. All of these cases will throw you out an error. Also there is another type of problem because of the strict rules listed above. For example when you are developing a RTF Template but couldn’t get any data in the output even though there should be a data and you have mapped the XML element data in the RTF Template. This can often be the case where you have typed the element name in lower case when the element name is presented in upper case in your XML data. Yes, XML Element name is case sensitive! So you need to make sure if the XML Element name is upper case or lower case (or combination) in the XML data file first then need to correctly type the name in your RTF Template.

Many cases when you start developing Data Template or RTF Template and get some strange error that you have no idea are caused by violations of the above rules. So let’s keep those rules in your mind when you work with Data Template and RTF Template.

Provided by Kanichiro Nishida, Consulting Manager, Oracle EPM & BI Consulting

Well, what can I say... It's been a long time. We've got involved in many of BI Publisher or Advanced Reporting related projects in past months and couldn't find a time to update the blog. I know that's such a BS, so let's move on.... ;-)

Today, I'll talk about the basic of XML as part of the 'Before you start RTF Template' series.

I’m sure you have already seen XML files and known what the files look like. Especially the BI Publisher technology is designed around XML so it’s almost impossible not to see XML files when you develop any report. However, some of you might not have spent a time to learn what really the XML is more than what it looks. So, let’s spend some time (not a lot, so don’t worry!) to visit the XML basic.

What is XML ?

Uploaded with plasq's Skitch!

XML stands for eXtensible Markup Language. It is a W3C standard, and it was designed to describe data and to focus on what data is while a similar language called HTML was designed to display data and to focus on how data looks. XML uses a Document Type Definition(DTD) or an XML Schema to describe the structure of XML documents. Now it’s more common to use XML schema than the DTD to describe the data. And BI Publisher also uses XML Schema when it’s needed.

What are they? (Terminology)

Instead of going through all the detail of XML, I’ll focus on what’s really important for BI Publisher report developers. First, what are those tags inside the XML file? When you open the XML files you can see a bunch of tags and all of them looks same. However, there is some difference among them and each of them has its own name. The names you want to remember are the followings.

- Element
- Attribute
- Value
- Namespace
- Root
- Parent
- Child
- Sibling
- CDATA

Element

Element name is actually the main part of the XML and it acts as something like metadata. For example with the above sample XML the is the element name. With BI Publisher you use this as a place folder name when you map into RTF Template.

Attribute

Attribute is an additional metadata that tag along with the Element and add more meaning to the Element. For example with the above sample XML, ‘age’ is the attribute, which describes age of the employee in the EMPLOYEE element. And you can have multiple attributes per Element.

And Element and Attribute are the main component of the XML and the backbone of the file.

Value

Value is literally the value. Both the Element and Attribute can have their own values. The Element value is surrounded by Element start tag and Element end tag. e.g. Peter. The attribute value must be presented with double or single quotes surrounded. e.g. Peter

Node

Node is similar to something like group, unit, or family. And node can contain another node or multiple nodes inside and each node can act as an unit inside the higher level of nodes. With above sample, DEPARMENT is a node and you can find EMPLOYEE node inside the DEPARTMENT node.

Namespace

Namespace is used to differentiate the elements that have same name but means different.

Root Element

Root is something like the ‘root’ that is used for Unix file system. It means a top level Element and there is only one in one XML file.

Parent/Child/Sibling

These are the terms to describe relationships of each element. And this acts as exactly same as your family hierarchy or file system. So the node that is at one level above from where you are is called Parent node while the ones that are at one level below are called Child node. With above sample ORACLE node is parent node for DEPARTMENT element and EMPLOYEE element is child node. And this is a relative concept, so while DEPARTMENT node is a child node for ORACLE node, DEPARTMENT node is at the same time a parent node for EMPLOYEE node.

CDATA

CDATA is used for a section for a comment out area, where you can type anything. Any XML standard processor will not process the content in this area. We’ll cover in the next section about XML standard rules but the content stored in this CDATA section doesn’t apply such rules so you can basically type anything here for your use. BI Publisher uses this area with Data Template to store SQL query.

Anybody can start developing RTF Template with MS-Word right away. You download Template Builder, which is a MS-Word Add-in, and install it, then load a sample XML data file or connect to BI Publisher Enterprise Server, now you can use Template Builder’s UI wizard to construct tables, charts, or even crosstabs just a matter of 10 seconds or so. (I mean, just to start with!) From there you can start modifying the layout and formatting by just using MS-Word’s native functionalities that most of us are already familiar with. You can add page header/footer, change font size, color, or change table column size, etc. And by using Template Builder you can add even calculations pretty easily. So until this point everything looks just a piece of cake.

But the real challenge starts from here when you have some ‘complicated’ requirements that you didn’t think they would be that complicated. For example you might want to do a conditional formatting based on values inside the group that you’re working on or even outside the group. Or you might want to use some functions to process some data values in the way you need to use for another processing. Or you might need to customize the charts to meet your specific requirements. The Template Builder has been evolving for the last 2 years or so and adding many extra features that have reduced significant amount of manual customization work we needed before. However, we still need to perform some type of manual customization to meet real world requirements especially for pixel perfect type of reports and forms.

For such customization, you can double click a text form field, which opens a Template Builder dialog box, click on the ‘Advanced’ tab, type any code includes BI Publisher code (xdo) and XSL/XPATH/FO codes. Or for Chart customization, you can double click the chart image, which again opens MS-Word dialog box, and click on the ‘Web’ tab, now you can customize the chart definition XML with mix of XDO/XSL/FO codes.

This is where many developers stop and scratch their heads, and wonder what the hell is this ? And you take a look at BI Publisher’s User’s Guide, Tim’s Blog, google some useful tips, and try & error, in order to achieve what it used to look simple to do once. And next time you realize it’s already a couple of weeks past and your project manager or customers keep asking you when they can be done or even if they can be really doable.

Welcome to the world of XML. All of the customization requires an understanding of XML technologies. Without a decent understanding of the technology you are going to just spend more time for nothing. However, with a correct and good amount of understanding of the technology the manual customization with XML codes doesn’t look that difficult anymore. In fact you can see the consistent logic behind it and can grasp easier. You can even find more functionalities with the XML technology, which enables you to enjoy much more flexible development of RTF Template.

Well, you don’t need to become a XML developer or anything. You just need to get a ‘decent’ amount of understanding. Or I would say just ‘Basic’ level of understanding first. And this basic understanding of XML technology makes a whole world of difference. I can tell you because that is what happened to me when I started working with this technology. This also happened to many of my colleagues from BI & Reporting background.

So from the next blog I’ll cover some of the basic XML technologies. If you can’t wait for the next blog, I’d recommend you to go ahead and check this W3School’s web site where you can find not only the overview of the technology also quick tutorials for you to learn.

The topics I’ll cover would be,

- XML
- XSL
- XPATH
- XSL-FO

So stay tuned and see you at the next blog !

Do you know there is a group in Oracle who specializes in delivery BI Publisher based solutions ? Yes, there is such group exists today! And we are the one who have been delivering many of successful BI Publisher related projects for the last 2 years and plus. We are from EPM & BI consulting group within Oracle North America consulting organization. Our mission is to deliver BI Publisher related projects successfully with high quality to our clients.

Over the last 2 years we have developed many internal advanced training materials for our consultants based on real world experience (not based on text book!) so that our consultants can keep their high level of expertise and provide the best service in this area.

We have also developed many BI Publisher related solutions, which includes BI Publisher Accelerator, Oracle Reports Migration, Content Management with BI Publisher, Advanced Delivery, just to name a few, and we have been delivering such solution implementations with many of our clients.

The clients we have helped to implement such BI Publisher related solutions includes industries such as Financial Service, Manufacturing, Media, Government, Retail, Pharmaceutical, Higher Education, Food & Beverage, etc.

Now we’re starting this blog to

1) Communicate with external community to obtain better understanding of both business & technical challenge today
2) Introduce our BI Publisher related solutions
3) Provide our best practice and lessons learned from our project base experience
4) Provide useful technical tips and troubleshooting

With that, we hope we can contribute to BI Publisher community.

Our focus is BI Publisher related solutions and best practice from implementation and project delivery point of view. I’m sure we’ll cover some of the product features and functionalities, but as most of you must know by now that Tim Dexter’s BI Publisher Blog is the best place for such information.

So long story short, this is our first blog and we want to contribute and want to have fun! So stay tuned!