Friday Aug 22, 2008

Model Driven Mural/MDM - Configuration Files

Having figured out how to use code generation from UML in Netbeans we'll now have to find out how a Mural MDM Application is configured before we can then move on to developing the templates for generating the configuration.

This article assumes that you are familiar with MDM. If you need to get a quick start with Mural MDM I'd recommend watching this tutorial.

The basic configuration files are highlighted in the project structure screen shot below:

Here's a short introduction to each file (in-depth information on MDM configuration files):

File name

Description

References Data Model

master.xml

configures Matching Process e.g. threshold for matches and potential duplicates.

No

mefa.xml

configures Matching Service, e.g. fields used for matching, normalisation of fields

Yes

midm.xml

configures Master Index Management Console (MIDM), e.g. search forms available and fields on results pages

Yes

midm-security.xml

containsweb application security for MIDM: authorised operations for user roles

No

object.xml

defines the MDM data model. Artefacts like jav API, database stcripts are generated form this file

Yes

query.xml

defines the queries that are used by the matching process and the MIDM

Yes

security.xml

configures security for the Java API/EJB of the MDM application

No

update.xml

defines update strategy used in the application (how the single best record is calculated)

Yes

validation.xml

configures validatiion components that are applied to data received by the MDM application

No

codelist.sql

defines list of values for fields (object.xml specifies which fields use a "codelist" or list of values)

Yes

Let's look at those files that contain configuration relevant to the data model in more detail.

object.xml

This file describes the data model of the MDM application.An Enterprise Object defined in MDM is a composite of objects. However the data model is limited to one parent and one level of child elements. A typical example would be a CustomerEO composed from Customer and Address as well as Phone childs. However it is not possible to further decompose the Adress object into a businessAddress and homeAddress object.

Let's have a look at some of the information in this file:

  • <nodes> elements define the data entities used in the application such as patient, address, phone number, visits in healthcare or customer, address, credit card in retail.

  • Its <tag> sub element defines the name of the entity

  • <fields> define the attributes of an entity including their name, type, size/length, part of primary key and list of values if appropriate (e.g. for a gender or title attribute)

  • Finally the <relationships> element defines the root entity and it's children

Most of the information in the file is a description of an object model in the OO sense – some of the information is database oriented (length of attribute, date format used by the objects ...) - which means most of the information required in objex.xml can be derived directly from a UML class diagram:

  • Classes and relationships (composition)

  • Attributes of classes, data types of attributes

Some other information like list of values, length can be added to UML components via tagged values or stereotypes.

midm.xml

Defines the appearance of the MDM management console. It consists of two main sections:

  • <node> element which is the object definition in terms of how it should be presented in the GUI, e.g. order on forms, type of control (TextBox, DropDownList etc.). This is very similar to object.xml above so it is clear how this information can be derived from UML also with the help of tagged values.

  • <gui-definition> lists all the different pages that will be available in the MIDM (Master Index Data Management) web application. Out of the box there is a start/dashboard type page plus a number of search form and result pages for the different searches MDM supports (alpha, phonetical, by Id). These page definitions indicate the attributes on the various pages and so refer back to the object model. Once again a UML model could indicate if attributes are searchable or should be displayed in result sets using tagged values or stereotypes.

mefa.xml

Defines which attributes should be normalized, standardized and/or phonetized. It also defines which attributes are used for matching records.

All of these sections define a set of attributes that should be transformed and mapped into another attribute. For example if first names are standardized the value in the first name attribute (Bill) will be transformed to its standardized form (William) and then stored in a “standardized first name” attribute.

A similar process (normalization) applies for free form texts for example addresses.

When matching data obviously these processed attributes should be used.

query.xml

Configures the query mechanisms used by MDM. There is some object model related data in here in the definition of blocker searches.

These blocker searches act as a crude set of filters for the match engine to identify possible duplicates for a record in a matching process. This is being used as a performance optimization – using block searches possible candidates are selected form the potentially large pool of data sets available. Then the matching engine can apply more sophisticated mechanisms to determine if the selected data sets constitue a possible match.

Attributes in the object model could be tagged as being used in such queries.

update.xml

Defines the calculation of the single best record (SBR) from the data supplied by different systems (System Records). Attributes of the object model are designated as being part of the SBR in this file.

codelist.sql

Should contain all the list of values referenced by “<code-module>” elements in object.xml. Lists are supplied in a straight forward way – codemodule name and values for the code module.

Part of this file could be derived from the UML model – attributes that use a list of values could be tagged as code-module.

Wednesday Jul 02, 2008

Model Driven Mural/MDM - NetBeans UML & Code Generation

Read this for background.

UML Class Diagram

Here's the UML example I will use for illustration purposes in the upcoming posts: it shows a composition of three classes representing a Patient. We'll revisit this diagram later once we know more about the MDM configuration.

There are a couple of noticeable things on this diagram:

  • The Patient class's stereotype is MDM_Root
  • One of the patient class's attribute is tagged as a primary key
  • Some of the attributes have tagged values like size or matchType
  • The classes Address and Phone are associated with the Patient class and this association is navigable.
I will explain the reasons for these additions to the model later on - for now the important thing to note is that the model contains this additional information to the basic class information.

Understanding NetBeans UML Code Generation

The first step on achieving Model Driven Development (MDD) bliss with Mural is to actually understand how the NetBeans template library and the UML module work.

Luckily somebody else already wrote an excellent article on the basics -NetBeans UML Custom Code Generation.

Let me just paraphrase the article before I'll add some additional points:

  • Templates used for code generation are Freemaker templates - so it's handy to have the link to the reference close by. There is also a Freemaker NetBeans Plug In referenced in the code generation tutorial.
  • There are a couple of NetBeans tools involved in the process the Template Manager, UML Options Panel (Templates tab) and UML Project Properties (Templates section). The article talks you through all these tools in detail.
    • Template Manager: add/manage template files used by different NetBeans Plug Ins. Here we will add our MDM templates.
    • UML Options Panel: here we will map our templates to a type of model component and UML stereotype
    • UML Project Properties: we will have to enable the templates we have created on a per project basis.
The article guides you through all the necessary steps to add a set of templates (EJB SLSB) to NetBeans. The interesting part here is that the way the templates are set up they are applied to classes with a certain stereotype. It looks like we will have to add our own steretype for MDM type elements. The sample templates are a bit more straightforward to understand than the templates shipped with the UML Plug In so this is a good starting point to gain an understanding how the code generation works.

Understanding the Source Data Model & Developing Templates

While the article makes a good job at describing the nuts and bolts of the tooling it doesn't actually explain how the EJB templates were developed in the first place. There are two areas one needs to know about in oder to understand and create templates:

  • Framemaker Scripting - hence the link above
  • Source Data Model - this is briefly described in the tutorial and the description mentions classInfo and modelElement

Knowing more about the latter will help tremendously in developing templates. Both classInfo and modelElement refer to the same UML object. They will contain the information from the model of the UML object (class, interface etc.) that code generation is run for. classInfo is an instance of class org.netbeans.modules.uml.integration.ide.events.ClassInfo. This class is actually an abstraction of the underlying UML model - it exposes a sub set of the available information that is deemed to be relevant for generating Java Code. It actually acts as a wrapper around another class that provides fine grained access to the underlying UML model.
Therefore every time one needs to access UML attributes like "tagged values" or "stereotype" or "primary key" one needs to get hold of an instance of the underlying fine grained class.
modelElement is just this - an instance of the underlying fine-grained class.

The same model applies to other elements like class attributes or methods. Here's a table of the most important classes. I have included links to the source code of these classes as this proved to be the best reference I could find (there is definitely no Javadoc for the simplified API and I couldn't locate a Javadoc for the fine-grained interfaces yet)

UML Type

 Fine Grained Representation

 Simplified Representation

Element

org.netbeans.modules.uml.core.metamodel.core.foundation.IElement

org.netbeans.modules.uml.integration.ide.events.ElementInfo

Class

org.netbeans.modules.uml.core.metamodel.infrastructure.coreinfrastructure.IClassifier

 org.netbeans.modules.uml.integration.ide.events.ClassInfo

 Attribute

 org.netbeans.modules.uml.core.metamodel.infrastructure.coreinfrastructure.IAttribute

org.netbeans.modules.uml.integration.ide.events.MemberInfo

 Method

org.netbeans.modules.uml.core.metamodel.infrastructure.coreinfrastructure.IOperation

org.netbeans.modules.uml.integration.ide.events.MethodInfo

Each of the Simplfied interfaces provides access to the underlying Fine Grained interface, e.g. ClassInfo offers a "getClassElement()" method just as MemberInfo provides a "getAttribute()" method.

Template Examples

Let's look at some template examples to see how these classes are used:

The example below renders some XML for the class that code generation is run on. I've highlighted how the classInfo itself is used, how the member variables/attributes of the class are accessed and how to get hold of the underlying object implementing IAttribute  to obtain Tagged Values that were added in the UML diagram.

   <nodes>
        <tag>${classInfo.getShortClassName()}</tag>
<#list classInfo.getFieldsCodeGenSorted() as fieldInfo > 
        <fields>
            <field-name>${fieldInfo.getName()!}</field-name>
            <field-type>${fieldInfo.getCodeGenType()?lower_case!}</field-type>
    <#if fieldInfo.getAttribute().getTaggedValueByName("size")?? >
            <size>${fieldInfo.getAttribute().getTaggedValueByName("size").getDataValue()!}</size>
    <#else>
            <size>20</size>
    </#if>
        </fields>
</#list>
    </nodes>

There are navigable relationships in our class diagram so here's an example how to follow these relationships (this is going to be important since all of the domain object model needs to be represented in MDM). All of the code below actually uses the fine-grained access to the UML model.

In the example we first obtain all relationships of the element then use the other end of the relationship to obtain information about the element/class that is associated with current element/class.

<#list modelElement.getAssociationEnds() as thisEnd > 
<#list thisEnd.getOtherEnd() as otherEnd > 
    <nodes>
        <tag>${otherEnd.getParticipant().toString()}</tag>
<#list otherEnd.getParticipant().getAttributes() as elem > 
        <fields>
            <field-name>${elem.toString()!}</field-name>
            <field-type>${elem.getTypeName()?lower_case!}</field-type>
        </fields>
</#list>
</#list>
</#list>


Tuesday Jul 01, 2008

A Model Driven Approach to using Mural/MDM

Some Background

In the context of my work for the OeHF an interesting idea came up from the foundation partners. OeHF is about building an open source framework for Healthcare applications. The core of such a framework is a lot of patient related data from diverse sources like e.g. patient demographics, visit data, medical records.
All of these sources apply their own local patient identifier and patient specific data  - somehow all these bits of data need to be matched and merged. The framework therefore requires to have some kind of Single Patient View tieing together all of this information. Sounds like Mural is a very good fit for these requirements.

So we were discussing how we could integrate Mural/MDM into the architecture and the question came up if it is possible to drive Mural from UML diagrams. Since we don't want to pass MDM objects all around our application the idea is to model the business objects and then derive a mural application from the classes.
As a consequence there's a requirement for a  Single Patient View adapter implementing a domain object centric interface and mapping the domain objects into the mural data model.

That potentially means a lot of Java mappings on quite similar and possibly complex data structures (in the case of OeHF the data structures will be aligned to the HL7 data model). So avoiding to code and maintain these mappings manually became another motivating factor for looking into how model driven development could be applied to Mural/MDM.

The Road Ahead

In a nutshell we'll need to come up with a way of generating Java code and XML configuration files from a NetBeans UML model. Ideally the resulting artefacts should be propagated into the appropriate projects and integrating the build cycles of these projects would be another nice to have.

Here's what we'd need to find out and do in order to make this happen:

  • Understand the UML code generation mechanism
  • Understand the MDM configuration files and which of these can be generated from the model
  • Develop a sample UML model and define required conventions for additional MDM related information required by the configuration files
  • Implement templates for the configuration files
  • Design and implement a solution to also generate the Java mappings from object model to MDM and vice versa
  • Possibly integrate generated component into build processes of the relevant projects
I'll tackle all the points above in subsequent posts to the blog in some form of step by step guide. So please bear with me.


So what's this all about then?

I am a German living in exile in the UK. Obviously that's the reason for the title and tagline which are of course borrowed from Fawlty Towers.

Like all the other guys in FAST - see Blogroll -  I am now on a mission to blog about JavaCAPS, openESB, NetBeans, Mural and so on. (Well, I have been ordered to do so and you know what it's like with Germans and orders)

I guess my first series of entries will be about the work I do in the context of the Open eHF - will be touching on NetBeans UML and code generation, Project Mural and openESB. And I'll also re-cycle some of the work I have done over the last few years into neat blog entries as well.

Apart from embarassing my girl-friend by speaking in German with a loud, harsh tone (is there any other way of speaking German?) while shopping at Tesco's I like to spend my personal time making and listening to music, reading and travelling. Also cooking and eating. Which is the reason why I developed a keen interest in running over the past year or so. So I might blog about some of this stuff as well.


About

Swen, resident alien in the UK based part of a team at SUN called FAST

Search

Top Tags
Categories
Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today