Mapping Key Value Pairs onto JSON Objects in Oracle Integration Cloud

We were recently doing some work on a system, actually a Robotic Process Automation (RPA) endpoint, that generated unique JSON messages for each type of request.  So a single interface would expect different objects, depending on the request.  The target system actually required a small orchestration to submit a single request and so ideally a single integration would abstract the interface to that service.

To help reify things, here is an example:

Request One Create Order

This is the request to be sent to the generic service

{
   "session" : "ABC123",
   "operation" : "createOrder",
   "data" : {
       "Customer" : "Antony",
       "Item" : "Stuffed Spinach Pizza"
   }
}

Note that the data has named fields.

Request Two Get Order

This is another request to be sent to the same service, but a different operation results in a different payload.

{
   "session" : "ABC123",
   "operation" : "getOrder",
   "data" : {
       "OrderID" : "112358",
       "FetchAllFields" : "True"
   }
}

Note that the operation has changed and as a result the named fields in data are now different.

So even though the endpoint is the same, and the call preparation and tear down are the same, it appears we will have to have a unique integration for each type of request.  That sounds like a lot fo duplicate work :-(

The Problem

We want to have a generic interface to our target, but because it takes different data formats for different operations that means custom coding for each operation.  Within OIC we need to define the data shape so that when we map data we know what fields we are mapping.

Generic Interface Solution

Ideally we would like to have a single interface with a generic interface, something like the one below:

{
   "session" : "ABC123",
   "operation" : "createOrder",
   "dictionary" : [
     {
       "key" : "Customer",
       "value" : "Antony"
     },
     {
       "key" : "Item",
       "value" : "Stuffed Spinach Pizza"     }
   ]
 }

Note the use of key value pairs that allow us to pass arbitrary data into the integration.  The only problem is how do we map the data.  We need to create entries in the target data object corresponding to the keys in the source dictionary and then set those entries in the data object to the corresponding value from the dictionary.

Partial Solution

If we knew the target fields then we could use array indexing to select the correct value corresponding to the name as shown below:

Note that this is using the new JET based mapper which will shortly be available in Oracle Integration Cloud, it is currently in controlled availability.

Here we select the dictionary item whose "key" is Customer and puts its value into the Customer field.  This doesn't work if we don't know the field names as is the case for us!

This works if we know the target names when we build the integration and using the map above we can transform data as shown below

{
   "session" : "ABC123",
   "operation" : "createOrder",
   "dictionary" : [
     {
       "key" : "Customer",
       "value" : "Antony"
     },
     {
       "key" : "Item",
       "value" : "Stuffed Spinach Pizza"
     }
   ]
}

{
    "session": "ABC123",
    "operation": "createOrder",
    "data": {
        "Customer": "Antony"
    }
}

Unfortunately we don't always know the names ahead of time in which case this solution doesn't work.

A Generic Solution

So lets look at a more generic solution.  There is an XSL <element> tag that can be used to create an arbitrary element.  Unfortunately we have to use this in a hand crafted XSL as the mapper does not support the <element> tag - yet.

The process is:

1. Map as much as you can using the mapper, including a single array lookup similar to the one above.
2. Export your integration
3. Unzip the integration
4. Find your XSL map in the unzipped package (it will be in the folder icspackage/project/YOUR_INTEGRATION_NAME_VERSION/resources/processor_XX/resourcegroup_YY where XX and YY are arbitrary numbers)
5. Edit the XSL map replacing the array mapping with the following:
   • Create a for loop over the base array element, the dictionary in our example
   • Create an element in the for-loop with name from key element and value from value element
     • <xsl:element name="{nsmpr0:key}"><xsl:value-of select="nsmpr0:value"/></xsl:element>
6. Import the XSL into the integration

The mapping before and after looks like this

<nsmpr0:response-wrapper xml:id="id_16">
   <nsmpr0:session xml:id="id_17">
      <xsl:value-of select="/nstrgmpr:execute/nsmpr0:request-wrapper/nsmpr0:session" xml:id="id_18"/>
   </nsmpr0:session>
   <nsmpr0:operation xml:id="id_19">
      <xsl:value-of select="/nstrgmpr:execute/nsmpr0:request-wrapper/nsmpr0:operation" xml:id="id_20"/>
   </nsmpr0:operation>
   <nsmpr0:data xml:id="id_24">

      <nsmpr0:Customer xml:id="id_25">
         <xsl:value-of xml:id="id_26" select="/nstrgmpr:execute/nsmpr0:request-wrapper/nsmpr0:dictionary[nsmpr0:key = &quot;Customer&quot;]/nsmpr0:value"/>
      </nsmpr0:Customer>

   </nsmpr0:data>
</nsmpr0:response-wrapper>

<nsmpr0:response-wrapper xml:id="id_16">
   <nsmpr0:session xml:id="id_17">
      <xsl:value-of select="/nstrgmpr:execute/nsmpr0:request-wrapper/nsmpr0:session" xml:id="id_18"/>
   </nsmpr0:session>
   <nsmpr0:operation xml:id="id_19">
      <xsl:value-of select="/nstrgmpr:execute/nsmpr0:request-wrapper/nsmpr0:operation" xml:id="id_20"/>
   </nsmpr0:operation>
   <nsmpr0:data xml:id="id_24">
      <xsl:for-each select="/nstrgmpr:execute/nsmpr0:request-wrapper/nsmpr0:dictionary">
         <xsl:element name="{nsmpr0:key}">
            <xsl:value-of select="nsmpr0:value"/>
         </xsl:element>
      </xsl:for-each>
   </nsmpr0:data>
</nsmpr0:response-wrapper>

Note that once imported into the integration you cannot edit it using the mapper in OIC.

JSon Note

The above example works fine because it is generating JSON and the XML/REST conversion in OIC does not pay attention to namespaces because there is no such construct in JSON.  If we wanted to do the same with XML output then we would nee to be more respectful of namespaces, although the element tag does support namespace specification.

Summary

We can deal with JSON data types that are unkown at design time by using Key Value pairs to dynamically construct the correct JSON objects.  This can be done in OIC and allows us to create generic integration wrappers to services that dynamically generate data types.